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ABSTRACT 


In  the  Federal  Acquisition  Regulations  (FAR)  Statement  of  Guiding  Principles  for  the 
Federal  Acquisition  System,  the  vision  of  the  Federal  Acquisition  System  is  to  deliver  best 
value  products  or  services  to  the  customer.  Contracting  Officers  must  achieve  this  while 
balancing  the  many  competing  interests  of  the  stakeholders  in  the  System.  The  paradox  of 
efficiency  vs.  effectiveness  can  be  found  in  the  second  sentence  by  the  phrase  “balancing  the 
many  competing  interests  in  the  System”.  This  statement  indicates  the  diverse  interest  of  the 
many  stakeholders  involved  in  the  System  that  in  many  instances  prevent  the  Contracting 
Office  from  being  efficient  and  effective.  The  Government  Performance  Results  Act  of  1993 
also  requires  each  agency  to  establish  projected  outcomes  or  results  by  which  they  will  be 
evaluated  against.  This  thesis  examines  various  literature  and  existing  measurement  systems 
of  field  contracting  offices  to  determine  if  we  are  properly  evaluating  efficiency  and 
effectiveness.  The  thesis  also  utilizes  the  Organizational  Configuration  Model  developed  by 
Nancy  Roberts  to  determine  where  field  offices  fit.  The  thesis  identifies  common  themes 
found  in  metrics  and  draws  conclusions  based  on  that  information.  Finally,  the  researcher 
proposes  a  model  for  Field  Contracting  Offices  to  use  for  evaluating  their  efficiency  and 
effectiveness.  It  is  the  researcher’s  hope  that  this  thesis  will  be  of  benefit  to  all  field 
contracting  offices  that  struggle  with  determining  their  efficiency  and  effectiveness.  In 
addition,  it  is  hoped  that  Systems  Commands  find  some  useful  information  in  this  thesis. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Why  is  efficiency  and  effectiveness  important  to  a  Contracting  Office?  In  the 
Packard  Commission’s  (1986  p.42)  final  report  to  the  President  it  states: 

We  must  give  acquisition  personnel  more  authority  to  do  their  jobs.  We 
must  make  it  possible  for  people  to  do  the  right  thing  the  first  time  and 
allow  them  to  use  their  common  sense.  When  this  is  done,  layers  of 
supervision  can  be  eliminated,  reporting  can  be  minimized,  and  the 
Department  of  Defense  (DOD)  can  get  by  with  far  fewer  people.  Only 
then  will  productivity  and  quality  become  hallmarks  of  defense 
acquisition. 

This  statement  gives  a  clear  indication  that  the  System  must  become  more  efficient. 
This  is  further  reinforced  in  the  Federal  Acquisition  Regulations  (FAR)  Statement  of  Guiding 
Principles  for  the  Federal  Acquisition  System,  namely,  the  vision  of  the  Federal  Acquisition 
System  states: 

All  participants  in  the  system  are  responsible  for  making  acquisition 
decisions  that  deliver  best  value  product  or  service  to  the  customer.  Best 
value  must  be  viewed  from  a  broad  perspective  and  is  achieved  by 
balancing  the  many  competing  interests  in  the  System.  The  result  is  a 
system  which  works  better  and  costs  less.  (FAR  1.102-1) 

The  paradox  of  efficiency  vs.  effectiveness  can  be  found  in  the  second  sentence  by 
the  phrase  “balancing  the  many  competing  interests  in  the  System”.  This  statement  indicates 
the  diverse  interest  of  the  many  stakeholders  involved  in  the  System  that  in  many  instances 
prevent  the  Contracting  Office  from  being  efficient  and  effective.  In  Chapter  II  this  will  be 
discussed  in  further  detail  when  efficiency  and  effectiveness  are  defined.  The  guiding 
principles  also  tells  us  how  to  achieve  efficient  operations: 

To  achieve  efficient  operations,  the  System  must  shift  its  focus  from  “risk 
avoidance”  to  one  of  “risk  management.”  The  cost  to  the  taxpayer  of 
attempting  to  eliminate  all  risk  is  prohibitive.  The  Executive  Branch  will 
accept  and  manage  the  risk  associated  with  empowering  local  procurement 
officials  to  take  independent  action  based  on  their  professional  judgment. 

(FAR  1.102-2) 
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In  other  words,  empower  the  Procurement  Contracting  Officer  (PCO),  and  then  trust 
him/her  to  do  the  right  thing.  The  guiding  principles  also  empower  the  PCO  “to  innovate  and 
use  sound  business  judgment”  when  a  policy  or  procedure  is  not  specifically  addressed  in  the 
FAR.  In  theory,  this  seems  to  give  the  PCO  wide  latitude  to  exercise  his  discretion  in  making 
business  decisions.  In  reality,  this  premise  is  largely  dependent  on  the  support  the  PCO 
receives  from  his  upper  management  and  agency  hierarchy.  As  will  be  discussed  later  in 
Chapter  II  this  support  has  a  major  impact  in  a  contracting  office’s  efficiency  and 
effectiveness.  Indeed,  the  Federal  Government  is  enamored  with  the  idea  of  an  efficient  and 
effective  acquisition  system.  The  decade  of  the  1990s  led  to  some  significant  legislation  to 
promote  a  more  efficient  and  effective  acquisition  system.  This  legislation  included: 
Government  Performance  Results  Act  (GPRA)  of  1993 
Federal  Acquisition  Streamlining  Act  (FAS A)  of  1994 
Federal  Acquisition  Reform  Act  (FARA)  of  1995 
Clinger-Cohen  Act  of  1996 
FAR  Part  15  Rewrite  of  1997 

All  of  these  legislations  sought  to  make  the  acquisition  system  more  efficient  and  effective 
by  focusing  on  commercial  practices.  However,  the  Federal  Government  cannot  completely 
adopt  all  of  the  practices  of  the  commercial  sector.  The  Procurement  Round  Table  (PRT) 
echoed  this  point  in  their  2001  report: 

The  Federal  Government  has  been  and  always  will  be  different  from  the 
commercial  sector.  Thus,  while  striving  to  make  the  Federal  Acquisition 
System  more  like  its  commercial  counterpart,  the  PRT  continues  to 
recognize  the  unique  constraints  that  are  imposed  on  a  public  sector 
organization.  No  matter  how  commercial,  competitive,  or  cost  effective 
the  Federal  Acquisition  Systems  becomes,  it  ultimately  will  still  be 
governed  by  public  policies — policies  that  are  driven  not  only  by 
economic  objectives,  but  also  by  social  and  political  considerations. 

Despite  these  limitations,  the  Federal  acquisition  system  must  remain 
properly  focused  on  mission  accomplishment.  (The  Federal  Acquisition 
System:  Transitioning  to  the  21st  Century  (p.6)) 

As  shown  above,  as  a  public  entity,  becoming  efficient  and  effective  is  important  to 
its  ultimate  stakeholder,  the  taxpayer. 

The  problem  of  measuring  efficiency  and  effectiveness  of  any  organization  seems  to 
span  the  entire  public  sector.  The  Government  Performance  Results  Act  (GPRA)  of  1993  put 
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forth  a  requirement  that  organizations  begin  to  measure  themselves  against  desired  outcomes. 
In  other  words,  their  performance  and  existence  will  be  measured  on  their  results. 
Implementation  of  the  Results  Act  has  faced  a  number  of  challenges  since  enactment.  These 
challenges  have  been  well  documented  in  various  GAO  reports  (GAO/T-GGD-96-136; 
GAO/T-GGD-97-83;  GAO/T-GGD-98-44;  GAO/T-GGD-98-66).  GAO  auditors 
acknowledge  that  it  will  take  time  to  orient  federal  managers  to  a  new  way  of  planning  and 
managing  and  develop  routines  that  enable  them  to  process  information  and  make  informed 
decisions  about  what  they  do  and  how  they  do  it.  (GAO/T-GGD-97-1 13).  Most  Defense 
activities  still  have  difficulty  linking  subjective  mission  goals  to  objective  outcomes. 

Now  that  we  have  looked  at  why  efficiency  and  effectiveness  is  important,  Chapter  II 
will  focus  on  what  we  mean  by  efficiency  and  effectiveness. 

The  purpose  of  this  thesis  is  to  analyze  the  metrics  a  Field  Contracting  Office  utilizes 
to  determine  their  effectiveness  and  efficiency.  In  addition,  a  metrics  model  will  be 
developed  and  recommended  for  implementation 

The  basic  goal  of  this  thesis  is  to  propose  a  model  that  a  contract  office  may  use  in 
determining  their  efficiency  and  effectiveness.  Included  in  that  will  be  an  analysis  of 
relevant  metrics  that  help  to  guide  organizations  to  achieve  them. 

B.  RESEARCH  QUESTIONS 

1.  Primary  Research  Question 

What  model  is  needed  to  measure  the  efficiency  and  effectiveness  of  a  Navy  Field 

Contracting  office? 

2.  Secondary  Research  Questions 

a.  What  is  the  difference  between  efficient  and  effective  and  why  should  we 
strive  for  both? 

b.  Where  do  most  Field  Contracting  Offices  fit  in  the  Organizational 
Configuration  model? 

c.  What  metrics  are  typically  used  to  measure  Field  Contracting  efficiency  and 
effectiveness  and  what  are  the  issues  and/or  limitations  associated  with  these 
metrics? 

d.  What  type  of  metrics  model  is  required  in  order  to  measure  the  efficiency  and 
effectiveness  of  a  Navy  Field  Contracting  Office? 
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C.  METHODOLOGY 

The  proposed  area  of  research  for  this  thesis  concerns  the  efficiency  and  effectiveness 
of  a  Navy  Contracting  Office.  Specifically,  the  research  will  attempt  to  define  the  concepts 
of  efficiency  and  effectiveness  as  well  as  propose  a  set  of  metrics  or  a  model  that  may  be 
used  in  measuring  the  effectiveness  and  efficiency  of  a  Navy  Field  Contracting  Office. 

This  study  will  require  an  extensive  review  of  existing  literature  as  well  as  previous 
studies  perfonned  on  this  topic.  In  addition,  it  will  research  the  Organizational  Configuration 
model  as  depicted  by  Professor  Nancy  Roberts  and  analyze  where  Contracting  Offices  fit  on 
the  Organizational  Model  and  recommend  ways  to  optimize  efficiency  and  effectiveness. 
Various  metrics  will  be  studied  to  determine  their  effectiveness  in  measuring  the  success  of  a 
contracting  organization.  Also,  the  research  will  look  at  the  benefits  of  the  balanced 
scorecard  approach  in  measuring  organizational  success.  The  researcher  is  in  the  unique 
position  of  being  responsible  for  establishing  a  metrics  program  and  quality  assurance 
program  at  the  Fleet  Industrial  Supply  Center  (FISC)  Norfolk  Detachment  Philadelphia,  and 
as  such  has  extensive  access  and  insight  into  the  current  process,  metrics,  and  other  data. 

The  theoretical  framework  in  which  the  study  will  be  performed  will  be  centered 
around  first  defining  the  organizational  configuration  model  of  the  contracting  office  as 
depicted  in  Nancy  Robert’s  research  on  Organizational  Configuration.  By  first  doing  this, 
we  can  examine  the  tradeoffs  between  efficiency  and  effectiveness  that  a  contracting  office 
should  consider.  The  researcher  will  then  examine  the  balanced  scorecard  approach  in 
measuring  organizational  success.  Finally,  the  researcher  will  propose  a  model  for 
measuring  the  effectiveness  and  efficiency  of  a  contracting  office.  The  researcher  plans  on 
using  FISC  Norfolk  Detachment  Philadelphia,  as  a  sample  of  many  of  the  contracting  issues 
that  effect  a  contracting  organization.  Additionally,  the  researcher  will  gather  information 
from  other  contracting  offices  as  well  as  look  at  several  private  sector  organizations  to 
detennine  a  benchmark  or  possible  best  practices  in  organizing  and  measuring  a  contract 
office. 

The  researcher  will  use  a  variety  of  methods  to  gather  data.  First  a  comprehensive 
review  of  the  literature  will  be  conducted  that  will  include  Department  of  the  Navy, 
Department  of  Defense,  other  Federal  Agencies,  and  commercial  sources.  This  data  will  be 
used  to  provide  background  on  the  current  state  of  measuring  organizational  performance. 
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The  researcher  will  also  analyze  a  summary  of  previous  Navy  Studies  conducted  over  the  last 
10  years.  These  studies  will  serve  to  identify  problem  areas  and  successes.  Policies,  best 
practices,  General  Accounting  Office  (GAO)  reports,  National  Association  of  Purchase 
Managers  (NAPM)  research  studies,  the  balanced  scorecard  approach,  professional  journal 
articles  and  previous  theses  will  be  reviewed,  incorporated  and  cited  as  applicable. 
Electronic  searches  via  the  Internet  will  be  a  primary  method  of  searching  the  literature. 

The  researcher  is  in  a  unique  position  at  FISC  Norfolk  Detachment  Philadelphia  that 
allows  access  to  metrics  data  gathered  at  all  levels  of  the  organization.  The  researcher  is 
currently  responsible  for  implementing  an  overall  quality  assurance  program  at  FISC  Norfolk 
that  is  used  by  all  of  its  detachments  (Philadelphia,  Washington,  Hampton  Roads,  and 
Norfolk  Naval  Shipyard).  In  addition,  as  the  Quality  Assurance  Program  manager,  the 
researcher  is  required  to  develop  a  metrics  program  that  adds  value  and  meets  the 
requirements  of  FISC  Norfolk  as  well  as  Headquarters,  Naval  Supply  Systems  Command 
(NAVSUP).  The  researcher  plans  on  using  resources  at  NAVSUP  to  help  define  the 
methodology  they  utilize  in  determining  relevant  metrics.  The  researcher  also  plans  on 
contacting  other  Navy  activities  to  understand  their  methodology  in  selecting  metrics.  This 
data  along  with  informal  interviews  and  personal  observation  will  allow  the  researcher  to 
provide  a  detailed  description  of  the  current  environment  at  FISC  Norfolk. 

The  goal  of  this  study  is  to  provide  a  variety  of  benefits  to  FISC  Norfolk  and  the 
Naval  Supply  Systems  Command  (NAVSUP)  and  maybe  the  Department  of  the  Navy  as 
a  whole.  First,  the  study  should  provide  FISC  Norfolk  with  a  framework  to  implement  a 
relevant  metrics  program.  This  should  also  provide  headquarters  with  a  framework  and 
an  understanding  of  the  how  their  subordinate  commands  measure  performance.  This 
will  hopefully  provide  NAVSUP  with  a  standardized  approach  that  they  can  implement 
to  the  entire  field  of  contracting  offices.  This  may  also  lead  to  NAVSUP  recommending 
to  the  Assistant  Secretary  of  the  Navy  (ASN)  to  implement  this  approach  Navy  wide. 

The  researcher  does  not  plan  on  using  a  survey.  Many  of  the  studies  analyzed  are 
recent  and  provide  many  anecdotal  summaries.  Given  the  researcher’s  experience  in  this 
area,  informal  interviews  with  other  contracting  offices  and  benchmarking  some  industry 
practices  will  provide  enough  infonnation  to  derive  conclusions. 
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Lastly,  the  study  may  identity  current  pathologies  in  the  performance  process  that 
can  lead  to  additional  research  in  this  area  to  gain  additional  efficiencies  and  process 
improvement 

D.  SCOPE/LIMITATIONS/ASSUMPTIONS 

The  general  thrust  of  the  study  is  to  identify  the  relevant  metrics  you  should 
examine  to  detennine  the  performance  of  a  Contracting  Office.  Problems  of  capturing 
non-value  added  measures  are  commonplace  in  today’s  contracting  organizations. 
Linking  metrics  to  strategic  mission  and  operations  is  disjointed.  Also,  establishing  a  tier 
metric  approach  is  sometimes  difficult  to  implement.  What  Headquarters  needs  is 
different  than  what  the  Commanding  Officer  (CO)  needs,  which  is  different  than  what  the 
first  line  supervisor  needs.  A  linking  mechanism  must  be  established  so  all  proponents 
understand  where  they  fit  in  the  overall  metric.  In  addition,  the  goal  of  all  metrics  is  that 
they  are  value  added  and  motivate  positive  performance.  These  issues  are  critical  to 
understand  when  you  are  discussing  the  perfonnance  of  a  contracting  office. 

This  thesis  will  focus  on  measuring  efficiency  and  effectiveness  of  a  Navy  Field 
Contracting  Office.  As  such,  major  system  acquisition  and  headquarters  contracting 
offices  are  not  included  in  this  study.  Since  the  complexity  of  work  is  so  divergent  in 
these  types  of  Contracting  Offices,  it  is  difficult  to  establish  a  specific  model  for  a 
headquarters  command,  since  their  metrics  would  be  significantly  different  than  a  field 
contracting  office. 

Another  limitation  on  this  thesis  is  that  the  researcher  is  recommending  a  specific 
model  for  a  field  contracting  office.  This  will  include  some  strategic  metrics,  but  only  as 
they  relate  to  the  Navy  Field  Contracting  Office  and  their  relationship  with  NAVSUP. 
The  researcher  will  not  attempt  to  implement  specific  strategic  metrics  for  the  Navy 
overall.  The  metrics  limitations  will  be  to  a  Navy  Field  Contracting  Office. 

E.  OVERVIEW  AND  ORGANIZATION 

This  thesis  is  divided  into  four  chapters.  This  chapter  has  provided  background 
concerning  the  growing  interest  in  contracting  efficiency  and  effectiveness.  In  addition, 
the  primary  research  question  and  secondary  questions  along  with  the  scope,  limitations 
and  methodology  have  been  discussed. 
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Chapter  II  defines  efficiency  and  effectiveness  and  utilizes  the  Organizational 
Configuration  model  as  depicted  by  Professor  Nancy  Roberts.  It  analyze  where 
Contracting  Offices  fit  on  the  Organizational  Model  and  recommend  ways  to  optimize 
efficiency  and  effectiveness.  Various  metrics  will  be  studied  to  detennine  their 
effectiveness  in  measuring  the  success  of  a  contracting  organization.  Additionally,  this 
chapter  will  also  include  an  extensive  review  of  existing  literature,  including  an  analysis 
of  the  balanced  scorecard  approach. 

Chapter  III  presents  data  gathered  via  informal  telephone  interviews  and  a 
contractor’s  (Booz- Allen  Hamilton)  summary  of  previous  studies  performed  by  or  for  the 
Navy.  In  addition,  the  researcher  will  examine  a  study  performed  by  the  Center  for 
Advanced  Purchasing  Studies  (CAPS)  concerning  effectiveness  of  a  purchasing  office. 
The  researcher  will  then  analyze  the  results  of  the  data.  This  chapter  will  also  provide  the 
current  metrics  that  FISC  Norfolk  and  NAVSUP  utilizes  in  measuring  their  field 
contracting  offices  and  analyze  the  usefulness  of  these  metrics  as  well  as  look  at  some 
current  DCMA  metrics  and  analyze  their  usefulness  in  a  field  contracting  office.  The 
researcher  will  also  discuss  the  implications  of  the  obtained  results  to  the  data  collection 
process  and  analyze  the  results  from  the  perspective  of  the  literature  review  and  the 
current  state. 

Chapter  IV  provides  conclusions  based  on  the  research  results,  a  model  for 
measuring  the  efficiency  and  effectiveness  of  a  contracting  office  and  identifies  areas  for 
further  study. 
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F.  SUMMARY 

Efficiency  and  effectiveness  is  found  almost  everywhere  when  discussing 
Government  operations.  The  question  of  how  we  achieve  it  and  how  we  measure  it  continue 
to  perplex  most  contracting  offices.  To  achieve  a  proper  balance  between  efficiency  and 
effectiveness,  a  contracting  office  must  know  their  stakeholders  and  must  be  able  to  balance 
their  interests  against  the  overall  mission  of  their  organization.  Through  the  research  efforts 
embodied  in  this  thesis,  a  model  will  be  proposed  that  field  contracting  offices  can  utilize  in 
measuring  their  efficiency  and  effectiveness. 

Lastly,  the  study  may  identify  current  pathologies  in  the  performance  process  that 
can  lead  to  additional  research  in  this  area  to  gain  additional  efficiencies  and  process 
improvement. 
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II.  CONTRACTING  OFFICE  EFFICIENCY  AND 
EFFECTIVENESS  BACKGROUND 


A.  OVERVIEW  OF  A  FIELD  CONTRACTING  OFFICE 

In  order  to  define  the  mission  of  a  Field  Contracting  Office,  the  mission  of  its 
headquarters  command  must  be  defined  first.  The  Naval  Supply  Systems  Command's 
(NAVSUP)  primary  mission  is  to  support  the  Navy,  Marine  Corps,  Joint  and  Allied 
Forces  with  high-quality  supplies  and  services,  whenever  and  wherever  those  supplies 
and  services  are  needed.  NAVSUP  delivers  combat  capability  through  logistics.  Navy 
NAVSUP  receives  its  Contracting  Authority  from  the  Secretary  of  the  Navy.  NAVSUP 
is  responsible  for  management  of  the  Navy  Field  Contracting  System  (NFCS),  which 
comprise  NAVSUP's  command  activities  as  well  as  all  other  Navy  field  activities 
delegated  contracting  authority  by  COMNAVSUP,  the  Head  of  the  Contracting  Activity 
(HCA)  for  the  NFCS.  NAVSUP’s  contracting  directorate  serves  as  the  HCA  principal 
staff  for  contracting  policy  matters,  operational  review  and  specific  approval  actions. 
Additionally,  NAVSUP  advocates  and  facilitates  activities  necessary  to  accomplish  the 
cultural  and  process  changes  entailed  in  effecting  the  Acquisition  Reform  (AR)  goals  of 
reducing  acquisition  costs  and  process  lead  times  while  improving  product  and  service 
availability,  performance  and  reliability  (http://www.navsup.navy.mil).  Therefore,  it  is 
NAVSUP’s  mission  to  facilitate  the  achievement  of  effectiveness  and  efficiency  within 
their  field  offices.  ASN  further  defines  NAVSUP’s  contracting  responsibilities  to  include 
contracting  for  supplies  and  services  throughout  the  Department  of  Navy  for  which  no 
other  contracting  activity,  office  or  command  is  delegated  contracting  authority.  This 
includes  contracting  for  Naval  (but  not  Marine  Corps)  activities  visual  information 
(motion  picture  and  videotape)  productions  through  the  Naval  Media  Center,  unless  an 
exception  is  granted  by  the  Chief  of  Naval  Operations  (N09C4)  (Naval  Acquisition 
Procedures  Supplement  5201.601(90)(c)  (2)). 

A  field  contracting  office  is  responsible  for  implementing  NAVSUP’s  contracting 

mission.  A  typical  field  contracting  office  supports  both  afloat  and  ashore  activities  and 

procures  all  types  of  supplies  and  services  to  support  those  customer  commands.  In 

addition,  NAVSUP’s  field  offices  obligate  approximately  6  billion  dollars  a  year  on 
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Supplies  and  Services  (Procurement  Management  Reporting  System  FY  02  Obligations). 
A  NAVSUP  field  contracting  office  receives  their  contracting  authority  via  an  instruction 
(NAVSUPINST  4200.81  (series)).  A  field  contracting  office  is  responsible  for 
negotiating,  awarding,  and  administering  contracts  in  addition  to  providing  other  related 
services  to  their  customers  on  issues  that  relate  to  the  myriad  of  regulations  that  govern 
the  acquisition  process.  The  goal  of  a  field  contracting  office  is  to  provide  their  customers 
with  the  best  value  products  and  services  required  to  carry  out  their  mission.  As  was 
stated  earlier  in  the  FAR’s  guiding  principles,  the  Contracting  Officer  must: 

a.  Satisfy  the  customer  in  terms  of  cost,  quality,  and  timeliness  of  the 
delivered  product  or  service. 

b.  Minimize  administrative  operating  cost. 

c.  Conduct  business  with  integrity,  fairness,  and  openness;  and 

d.  Fulfill  public  policy  objectives. 

All  of  these  requirements  have  an  impact  on  the  efficiency  and  effectiveness  of  a 
contracting  office.  There  are  constant  tradeoffs  that  take  place  in  order  to  satisfy  the 
many  stakeholders  involved  in  the  process. 

NAVSUP  (as  is  the  rest  of  the  Navy)  is  currently  undergoing  a  significant 
“Transformation”  in  business  affairs  to  help  reduce  overall  Navy-wide  infrastructure 
costs  by  $10  Billion  dollars.  In  the  Jan/Feb  2003  Navy  Supply  Corps  Newsletter,  there 
appeared  an  article  on  “Transforming  NAVSUP”.  The  article  states,  ’’NAVSUP’  has 
embarked  on  a  journey  to  better  structure  and  align  the  organization  to  most  effectively 
and  efficiently  deliver  combat  capability  through  logistics”.  In  the  Fleet  Industrial 
Supply  Center  (FISC)  Norfolk’s  (the  researchers  parent  command)  presentation 
“Planning  and  Implementing  NAVSUP  Transformation”,  they  list  the  Transformation 
objectives: 

Maintain  ability  to  accomplish  mission. 

Seek  opportunities  to  improve  processes. 

Gain  efficiencies,  which  lead  to  reduced  costs. 
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The  goal  in  transformation  is  to  become  more  efficient  and  effective  by 
transforming  the  way  we  do  business.  There  are  several  working  groups  analyzing  all  of 
NAVSUP’s  functions  to  determine  how  to  transform  operations.  Unfortunately,  to  the 
best  of  the  researchers  knowledge,  productivity  or  efficiency  of  a  contracting  office  is  not 
being  analyzed,  only  actions,  commodities  and  geographical  location. 

Now  that  the  mission  of  a  field  contracting  office  has  been  defined,  the  question 
of  what  is  efficiency  and  effectiveness  needs  to  be  answered.  In  Chapter  III,  NAVSUP’s 
current  metrics  for  measuring  the  performance  of  their  field  contracting  offices  will  be 
analyzed  to  determine  if  they  are  truly  measuring  the  effectiveness  and  efficiency  of  their 
field  contracting  organizations. 

B.  LITERATURE  REVIEW  ON  EFFICIENCY  AND  EFFECTIVENESS  IN 

PERFORMANCE  MANAGEMENT 

The  acquisition  workforce  has  never  had  to  account  for  its  contribution  to  the 
overall  defense  strategy  of  the  United  States  in  an  explicit,  thoroughly  rigorous  manner. 
Since  the  end  of  the  Cold  War,  however,  the  acquisition  workforce  has  been  pressured  to 
justify  itself  both  in  tenns  of  value  creation  and  unique  competency  (Barzelay  and 
Thompson,  2001).  The  real  problem  seems  to  be  that  the  acquisition  workforce  has 
trouble  explaining  how  they  provide  value.  The  importance  of  defining  how  a  contracting 
office  provides  value  to  its  customers  cannot  be  understated.  It  is  the  crux  for  developing 
relevant  and  important  metrics.  This  leads  the  contracting  office  to  define  what  they  do 
and  why  they  do  it.  Value  analysis  requires  you  to  seek  ways  to  improve  cycle  efficiency 
(value-added  time  vs.  total  time)  and  determine  ways  to  reduce  associated  costs  due  to 
delays,  excesses,  and  non-value  added  processes  (Barzelay  and  Thompson,  2001). 

The  Federal  Computer  Week  (May  2003)  issued  an  article  explaining  the  current 
administration’s  desire  for  a  perfonnance-based  system.  According  to  Mark  Froman,  the 
administrator  of  the  office  of  E-Government  at  the  Office  of  Management  and  Budget, 
“The  White  House  will  continue  to  scrutinize  agencies  for  measurable  perfonnance  and 
increased  efficiency”.  The  administration’s  proposed  2004  budget  ties  agencies  funding 
increasingly  tightly  to  achievements.  The  2004  budget  specifically  addresses  five  areas 
of  continued  weakness,  he  said: 
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•  Automating  inefficient  processes,  or  "paving  cow  paths."  Simply 
converting  a  badly  planned  process  from  manual  to  automatic 
won’t  make  the  agency  more  effective.  Agencies  should  rethink 
their  processes  and  discover  how  to  get  the  most  from  available 
technologies. 

•  Program  management.  Far  too  many  programs  still  come  in  past 
deadline  and  over  budget. 

•  Redundant  buying.  Agencies  should  find  ways  to  consolidate 
operations  so  that  they  need  fewer  systems. 

•  Islands  of  automation.  Agencies  still  haven't  solved  the  age-old 
stovepipe  problem. 

•  Cybersecurity.  Keeping  information  protected  while  making  it 
available  to  those  authorized  to  have  it  is  a  never-ending  concern. 

While  the  majority  of  his  comments  were  direct  to  Infonnation  Technology  (IT) 
investments,  they  also  have  relevance  for  non-IT  investments.  Most  field  contracting 
offices  that  were  researched  found  the  same  problems  in  their  organizations,  especially 
the  first  and  third  bullets  cited  above. 

Graves  (2001)  defines  Government  efficiency  as  the  ratio  of  outputs  to  inputs.  It 
seems  simple  enough.  However,  the  general  lack  of  markets  to  assign  value  to  public 
sector  outputs  make  the  process  extremely  difficult.  Public  sector  pursuit  of  efficiency 
requires: 


Strategic  planning  to  infer  required  outputs  from  politically 
determined  outcomes  and  to  make  tradeoffs  between  the  desired 
outputs  to  determine  target  levels  of  service; 

Accounting  that  allows  quantifying  the  input  cost  that  contributes 
to  an  output; 

Quantitative  measure  of  output  goals  and  performance;  and 

Motivation  and  resources  to  establish  the  management  systems  and 
to  make  the  choices  that  maximize  efficiency. 

In  Lloyd’s  (Government  Contracting  Pathologies,  2000),  he  indicates  that  we 
should  calculate  whether  some  contracting  offices  have  excess  capacity.  If  so,  transfer 
personnel  to  make  more  sensible  distribution  of  workload.  But  how  can  a  detennination 
of  excess  capacity  be  done  without  an  analysis  of  a  contracting  office’s  efficiency. 
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In  Chapter  I  we  mentioned  many  of  the  acquisition  refonn  legislation  that  was 
implemented  in  the  1990s.  The  main  consideration  in  acquisition  reform  is  whether  the 
new  process  enables  us  to  field  better  weapon  systems,  faster,  and  cheaper  (Booher, 
Pinker,  Smith  1997).  This  suggests  that  we  must  become  more  efficient  in  our  operations 
and  processes.  In  addition,  in  order  to  attain  this  objective  and  demonstrate  that 
improvement  and  some  efficiency  has  been  achieved  it  is  imperative  that  we  establish 
some  measurement  for  comparing  the  old  process  to  the  new. 

(Newcomer  and  Scheirer,  January  2001)  studied  23  federal  agency’s 
implementation  of  the  Government  Performance  Results  Act  (GPRA).  In  their  report, 
they  identified  many  different  steps  to  performance-based  management.  They  found  that 
performance  based  management  is  much  more  than  simply  recording  measures  of 
program  performance  and  reporting  them  upwards  to  oversight  bodies  and  stakeholders. 
The  following  steps  are  needed  to  develop  and  collect  performance  measures  useful  for 
decision-making: 

Program  stakeholders  must  come  together  to  reach  agreement  on 
strategic  and  perfonnance  objectives  and  the  strategies  for 
achieving  them. 

Indicators  must  be  defined  for  program  components  that  capture 
program  outputs  and/or  outcomes. 

Data  sources  must  be  developed  or  discovered  for  those  indicators 
and  data  must  be  aggregated  and  reported  in  user-friendly  formats. 

Data  must  be  used  by  decision  makers  to  assess  and  improve 
results  and  the  data  must  be  addressed  at  every  step  in  the  process 
from  original  collection  to  final  reporting. 

In  addition,  evaluators  must  have  the  skill  set  to  accomplish  all  of  these  steps. 
This  requires  significant  analytical  capacity.  Unfortunately,  the  Newcomer  and  Schierer 
study  found  that  few  organizations  felt  comfortable  about  their  agency’s  capacity  for 
evaluation.  Most  of  this  was  due  to  the  fact  that  significant  parts  of  the  “evaluation  type 
people”  were  downsized,  as  they  were  not  considered  “production  type  employees”.  So 
while  GPRA  has  required  us  to  develop  performance/outcome-based  measures,  most 
organizations  have  lost  the  personnel  resources  to  do  this  type  of  work.  In  their 
recommendations,  they  offered  “the  system  must  strengthen  the  evaluation  skills  of  their 
workforce  as  the  basic  tenet  to  enhance  the  likelihood  that  the  performance  management 
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framework  being  institutionalized  via  GPRA  will  result  in  both  improved  program 
management  and  desired  results”. 

In  Paul  O’Connell’s  study  of  the  new  York  Police  department’s  CompStat  system 
(Using  Performance  data  for  Accountability,  August  2001)  he  found  a  sophisticated 
performance  measurement  system  that  reorders  an  organization’s  day-to-day  operations, 
as  well  as  its  overall  orientation  toward  its  core  mission  and  goals.  CompStat  is  a  metrics 
tool  that  gathers  various  stakeholder  input.  It  is  based  upon  the  compilation,  distribution 
and  utilization  of  “real  time”  data  in  order  to  allow  field  managers  to  make  more 
informed  and  effective  decisions.  The  author  awards  praise  to  the  CompStat  model 
because  it  included  many  stakeholders  in  its  development  and  allowed  key  stakeholders 
to  recommend  specific  measurement  tools  that  would  motivate  perfonnance.  It  is  widely 
recognized  as  a  significant  reason  for  the  reduction  in  serious  crime  in  the  New  York 
area.  However,  O’Connell  cautions,  “The  CompStat  process  is  not  a  panacea.  It  is  not  a 
magical  cure-all  that  will  transform  a  poorly  run  and  inefficient  organization  into  a  model 
of  public  service  excellence.  Rather,  it  is  an  additional  tool  that  can  be  used  to  enhance 
performance  by  means  of  careful  measurement  and  planning  and  effective  allocation  of 
resources”.  The  lesson  to  leam  here  is  to  identify  all  of  the  key  stakeholders  and  allow 
them  to  recommend  metrics  that  not  only  satisfy  the  strategic  direction  of  the 
organization,  but  also  motivates  performance  and  awards  successes. 

Frederickson  (The  Potential  of  the  Government  Performance  and  Results  Act  as  a 

Tool  to  manage  Third-Party  Government)  says,  “GPRA  seeks  to  improve  federal 

agency’s  efficiency  and  effectiveness”.  GPRA  posits  that  federal  perfonnance 

shortcomings  are  primarily  managerial,  specifically  attributable  to  poorly  articulated 

missions  and  inadequate  perfonnance  information.  Even  if  we  better  define  our  missions 

and  track  performance  to  that  mission,  our  efforts  can  be  curtailed  by  the  fact  that  the 

third  parties  (customers,  Congress,  taxpayers)  with  whom  we  partner  to  deliver  public 

services  are  not  uniform  in  either  kind  of  responsibility,  or  in  other  words,  they  do  not 

share  our  values  or  mission.  Frederickson’s  research  also  recommended  that  when 

developing  performance  goals,  agencies  should  make  clear  their  role  in  the  delivery  of 

public  service,  and  should  use  GPRA  not  only  as  a  means  to  communicate  their 

performance,  but  also  to  communicate  constraints  that  inhibit  their  performance.  It 
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would  be  interesting  to  report  to  Congress  (a  requirement  of  GPRA  is  to  coordinate  with 
key  stakeholders)  that  they  are  one  of  the  major  inhibitors  to  your  agency  in  achieving 
your  performance  goals.  Frederickson  also  provided  a  summary  of  the  recurring  themes 
that  the  Government  has  encountered  in  previous  attempts  to  implement  a  GPRA  like 
system: 

These  themes  include: 

It  is  difficult  to  reach  agreement  on  goals  and  to  find  adequate 
measures  to  determine  the  attainment  of  goals  even  if  there  is 
agreement  on  what  those  goals  should  be; 

Managers  often  turn  to  activity  and  output  measures  as  proxies  for 
outcome  measures; 

Performance  measurement  and  budgeting  represent  the 
superimposition  of  a  managerial  structure  on  a  political  process; 

While  perfonnance  information  has  not  proven  useful  for 
appropriators,  it  has  shown  some  promise  for  management 
decisions. 

Given  the  previous  attempts  at  reforms  similar  to  GPRA,  one  might  expect  a 
similar  result.  However,  Frederickson  contends  that  all  other  refonns  were  done  via  the 
executive  branch,  while  GPRA  originated  in  the  legislative  or  oversight  branch.  In 
addition,  GPRA  carries  the  force  of  law.  Even  if  agencies  thought  it  was  another  fad, 
they  are  legally  bound  to  comply  with  its  requirements.  Frederickson’ s  research 
answered  the  question  on  how  some  agencies  are  implementing  GPRA.  He  found,  based 
on  a  preliminary  review  of  the  FY  2001  performance  plans,  with  their  emphasis  on 
activity  measures,  rarely  did  an  activity  answer  the  question  of  Are  these  measures 
adequate  to  evaluate  the  achievement  of  program  objectives?  However,  he  points  out, 
given  the  methodological  and  logistical  problems  associated  with  measuring  a  public 
program’s  perfonnance,  it  is  not  difficult  to  understand  why  the  majority  of  and  agencies’ 
measures  reflect  inputs  and  activities. 

So  just  by  making  it  mandated  by  law  does  not  provide  us  a  solution  to  the 
problem.  As  the  literature  review  has  indicated,  we  are  spending  a  lot  of  time  trying  to 
figure  out  how  to  comply  with  GPRA  despite  the  fact  it  has  been  a  law  for  10  years. 
Agencies  are  constantly  wrestling  with  contradictory  Congressional  messages.  On  one 


15 


hand  you  are  held  accountable  for  your  results.  On  the  other  hand,  you  are  constantly 
being  pushed  to  devolve  more  responsibilities  to  lower  levels  of  government  or 
administer  them  through  private  entities  and  hold  them  responsible.  Probably  the  best 
lesson  to  learn  from  Fredrickson’s  research  is  that  GPRA-driven  reform  efforts  can 
accomplish  refonn  only  to  the  extent  that  they  do  not  conflict  with  authorizing  law  and 
other  existing  laws.  In  his  research  he  found  that  all  of  the  agencies  he  was  studying 
were  impeded  in  their  efforts  to  achieve  their  goals  and  collect  data  to  determine  the  level 
of  goal  achievement  by  existing  law. 

Leading  organizations  use  their  performance  management  systems  to  achieve 
results,  accelerate  change,  and  facilitate  communication  throughout  the  year  so  that 
discussion  about  individual  and  organizational  performance  are  integrated  and  ongoing. 
Thus,  an  effective  performance  management  system  can  be  a  strategic  tool  for 
organizations  to  drive  internal  change  (efficiency)  and  achieve  external  results 
(effectiveness)  (GAO  02  -966  Result  Oriented  Culture).  GAO  found  that  performance 
management  systems  in  leading  organizations  typically  seek  to  achieve  three  key 
objectives: 

1.  Provide  candid  and  constructive  feedback  to  help  individual 

employees  maximize  their  contribution  and  potential  in 
understanding  and  realizing  the  goals  and  objectives  of  the  agency. 

2.  Provide  management  with  the  objectives  and  fact-based 

information  it  needs  to  reward  top  performers. 

3.  Provides  the  necessary  information  and  documentation  to  deal  with 
poor  performance. 

Performance  management  has  to  start  form  the  top  in  order  for  the  system  to  operate  as 
designed.  Recognizing  this,  the  Office  of  Personnel  Management  (OPM)  amended 
regulations  on  governing  perfonnance  appraisals  for  Senior  Executives  requiring 
agencies  to  establish  performance  management  systems  that: 

1 .  Hold  senior  executives  accountable  for  their  individual  and 

organizational  perfonnance  by  linking  performance  management 
with  the  results-oriented  goals  of  the  GPRA. 

2.  Evaluate  senior  executive  perfonnance  using  measures  that 

balance  organizational  results  with  customer  satisfaction, 
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employee  perspectives,  and  any  other  measures  agencies  decide 
are  appropriate. 

3.  Use  performance  results  as  a  basis  for  pay,  awards,  and  other 
personnel  decisions. 

Agencies  were  to  establish  these  performance  management  systems  by  their  2001 
senior  executive  performance  appraisal  cycle. 

GAO  found  that  most  agencies  have  taken  the  first  step  in  developing  an  initial  set 
of  expectations  for  senior  executives  to  address  in  their  individual  performance  plans  that 
are  intended  to  balance  accountability  for  organizational  results  with  a  focus  on  customer 
satisfaction  and  a  consideration  of  employee  perspective.  In  their  report  to  Congress 
(GAO  02  -966  Result  Oriented  Culture;  Using  Balanced  Expectations  to  Manage  Senior 
Executive  Performance),  GAO  summarized  the  following: 

There  is  a  growing  recognition,  driven  by  a  variety  of  worldwide  trends 
and  pressing  long-tenn  fiscal  challenges,  that  the  federal  government  is  on 
the  brink  of  an  enormous  transformation  in  what  the  Government  does, 
how  it  does  business  and  in  some  cases  who  does  the  Government’s 
business.  Ultimately  successful  organizations  understand  that  they  must 
often  change  their  culture  to  successfully  transform  themselves,  and  that 
such  a  change  starts  with  top  leadership.  Senior  executive  perfonnance 
and  accountability  for  change  management  will  therefore  be  critical  to  the 
success  of  the  federal  government’s  transformation.  A  specific 
performance  expectation  to  lead  and  facilitate  change  could  be  a  critical 
element  as  agencies  transform  themselves  to  succeed  in  an  environment 
that  is  more  results-oriented,  less  hierarchical  and  more  integrated. 

The  lesson  to  learn  here  is  that  there  will  be  many  influences  involved  in 
determining  an  activity’s  performance  measurement  system,  some  of  which  will  be 
political  and  some  that  may  possibly  be  selfish. 

The  Veteran’s  Benefits  Administration  (VBA)  adopted  a  balanced  scorecard 
approach  in  fiscal  year  1999  as  a  strategic  management  tool  to  drive  organizational 
change,  provide  feedback  to  employees  on  measures  they  can  influence,  link  performance 
appraisal  and  reward  systems  to  perfonnance  measures,  and  provide  incentives  for 
managers  to  work  as  teams  in  meeting  performance  measures.  Its  scorecard  included 
measures  for  accuracy,  speed  and  timeliness,  unit  cost,  customer  satisfaction,  and 
employee  development  and  satisfaction.  VBA  incorporated  these  measures  in  the 
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performance  appraisals  for  senior  executives  in  their  regional  offices  where  the  majority 
of  senior  executives  are  located.  They  have  experienced  some  success  with  this 
approach.  Later  on  in  this  chapter,  the  balanced  scorecard  approach  will  be  analyzed  in 
detail  as  an  approach  to  achieve  performance  management  success  (GAO  02  -966  Result 
Oriented  Culture). 

Effective  performance  management  systems  seek  to  provide  candid  and 
constructive  feedback  to  help  individual  employees  maximize  their  potential  in 
understanding  and  realizing  organizational  goals  and  objectives  and  help  reward  top 
performers.  In  recognizing  the  importance  of  creating  a  results-oriented  culture  in  federal 
agencies,  Congress  is  considering  legislative  proposals  to,  among  other  things,  focus 
attention  on  the  impact  poor  performance  can  have  on  the  effectiveness  of  an 
organization  and  require  agencies  to  have  chief  human  capital  officers  to  select,  develop 
and  manage  a  productive  high-quality  workforce.  (GAO  02-862  Results-Oriented 
Culture;  Insights  for  U.S.  Agencies  from  other  Countries’  Performance  Management 
Initiatives)  However,  this  still  does  not  answer  the  question  of  what  you  can  do  about 
poor  performers.  Until  the  Govermnent  relaxes  their  performance  appraisal  system  to  be 
more  in  line  with  the  private  sector,  the  Government  will  continue  to  experience  the  poor 
performance  problem  with  limited  to  no  recourse.  Legislative  action  has  to  take  the  place 
of  discussing  what  we  do  with  poor  performers. 

C.  ORGANIZATIONAL  CONFIGURATION  MODEL 

Professor  Nancy  Roberts  has  done  extensive  research  in  the  area  of 
Organizational  efficiency  and  effectiveness.  Robert’s  perspective  is  that  all 
Organizations  are  configured  with  attributes  toward  efficiency  and  effectiveness.  The 
configuration  approach  is  a  “holistic  stance”  and  assumes  that  the  parts  of  an  organization 
“take  their  meaning  from  the  whole  and  cannot  be  understood  in  isolation”.  The 
configurations  represent  a  clustering  of  organizational  attributes  (e.g.,  environment, 
strategy,  structure,  culture,  beliefs,  processes)  that  fall  into  coherent  patterns.  The 
exercise  of  configuring  an  organization  first  starts  by  identifying  the  basic  two 
dimensions  of  organizational  performance  -  efficiency  and  effectiveness. 
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1.  What  Is  the  Difference  Between  Efficiency  and  Effectiveness  and  Why 
Should  One  strive  for  Both? 

Efficiency  refers  to  the  capacity  to  produce  results  with  the  minimum  expenditure 
of  time,  money,  or  materials  (Websters  1971,  725).  Efficiency  thus  focuses  on  the  input- 
output  ratio.  To  be  efficient  is  to  do  things  well,  to  attend  to  the  internal  organization  by 
refining,  formalizing,  and  elaborating  on  existing  knowledge  and  making  short-term 
improvements.  Effectiveness  on  the  other  hand,  is  defined  as  “productive  of  results” 
(Websters  1971,  724).  The  focus  is  on  doing  the  right  thing  and  that  is  determined  by 
outcome  attainment.  It  comes  from  an  understanding  and  interpretation  of  the  exterior 
environment  and  what  outcomes  are  required  (e.g.,  Customer  satisfaction,  goal 
achievement,  negotiated  savings). 

As  discussed  above,  both  efficiency  and  effectiveness  play  an  important  role  in 
organizational  perfonnance,  yet  in  the  competition  for  resources,  each  can  interfere  with 
the  other.  Efficiency  depends  on  focus,  precision,  repetition,  analysis,  discipline  and 
control.  Effectiveness,  on  the  other  hand,  relies  on  experimentation,  novelty  and 
loosened  control.  While  effectiveness  thrives  on  experimentation,  efficiency  attempts  to 
drive  out  any  uncertainties.  Consequently,  public  managers  need  to  make  choices 
between  the  level  of  efficiency  and  effectiveness  they  intend  to  pursue.  Figure  1 
illustrates  the  four  possible  combinations,  identified  as  configurations  or  ideal  types  that 
can  be  theoretically  derived  when  public  sector  managers  seek  different  levels  of 
efficiency  (horizontal  dimensions)  and  different  levels  of  effectiveness  (vertical 
dimension). 

Efficiency  looks  at  internal  operations,  while  effectiveness  is  viewed  as  how  well 
are  we  performing  by  the  external  environment  (stakeholders).  An  Organization  can  be 
considered  quite  effective  by  a  customer,  yet  be  significantly  inefficient  when  you 
examine  their  internal  operating  procedures. 

So  why  do  we  need  to  strive  for  both?  Simply  stated,  we  are  required  to.  As 
stated  previously  in  this  chapter  “NAVSUP  has  embarked  on  a  journey  to  better  structure 
and  align  the  organization  to  most  effectively  and  efficiently  deliver  combat  capability 
through  logistics”.  In  addition  to  Navy  transformation,  the  goal  of  the  GPRA  is  to 
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measure  us  by  our  outcomes  and  demonstrate  to  the  taxpayer  that  we  are  an  efficient  and 
effective  organization. 


HIGH 


EFFECTIVENESS 


LOW 


Figure  1.  Organizational  Configurations 

2.  Directive  Configuration 

The  Directive  Configuration  promotes  optimal  efficiency  with  less  attention 

devoted  to  effectiveness.  In  this  configuration,  public  managers  pursue  efficiency  by 

running  their  organizations  like  well-oiled  machines.  They  focus  on  maintaining  internal 

order,  thus  attempting  as  much  as  possible  to  cut  the  organization  off  from  any  disruptive 

external  influences.  Managers  insist  on  a  common  direction  to  set  goals  and  objectives  in 

order  to  keep  members  accountable.  They  require  formalized  jobs  and  standardized  work 

to  maintain  an  orderly  coordinated  activity.  They  oversee  uniform  policies  that  cover 
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rights  and  duties,  promotions  based  on  competence  and  merit,  and  an  impersonal 
relationship  with  the  workforce  to  ensure  the  smooth  flow  of  work.  They  use  budget  and 
operational  controls  to  monitor  actions  and  most  organizational  change  is  geared  to 
improve  operations.  Since  change  can  disrupt  orderly  routines,  they  minimize  it  unless  it 
is  deemed  essential.  When  forced  to  make  a  change,  managers  take  on  the  role  of 
strategic  planner  and  driver  of  the  organizational  system  by  using  top-down  directives  to 
modify  standard  operating  procedures.  The  management  direction  setting  and  planning  is 
a  top-down  approach  that  sets  goals  and  strategy  based  on  increasing  efficiency.  The  role 
of  the  organizational  manager  is  a  planner/controller  type  and  the  role  of  the  employee  is 
“soldier”  and  order  taker.  Discretion  and  flexibility  are  minimal  in  deference  to  the 
nonns  of  order,  stability,  and  respect  for  higher  authority. 

3.  Responsive  Configuration 

The  Responsive  Configuration  relieves  the  tension  between  efficiency  and 
effectiveness  by  ignoring  both  dimensions.  What  this  means  in  practice  is  that  public 
managers  do  not  push  their  organizations  toward  effectiveness  or  efficiency,  nor  do  they 
make  much  effort  to  reconcile  the  competing  demands  between  the  two.  This  is  crisis 
management  at  its  best  as  managers  muddle  through  the  decision  making  process.  We 
see  this  type  of  organization  typically  in  the  political  minefields.  In  their  attempts  to  be 
responsive  to  external  political  forces,  they  are  forced  to  make  a  choice  between  crisis 
manager  (douse  political  fires  when  they  erupt  to  accommodate  contentious  stakeholders) 
and  power  brokers  (join  in  the  fray  by  building  a  base  of  power  and  develop  strategies  to 
protect  organizational  interests).  Whether  they  choose  the  role  of  crisis  manager  or 
broker  of  power  organizational  policies  are  the  result  of  responses  made  to  competing 
demands  made  in  the  political  arena.  Research  evidence  to  support  the  Responsive 
Configuration  comes  from  many  sources.  In  the  business  literature,  organizations  falling 
into  this  configuration  are  often  treated  as  reactors — poor  perfonners — because  they  are 
considered  neither  effective  nor  efficient.  Alternatively,  in  a  highly  regulated 
environment  organizations  falling  into  this  configuration  are  viewed  as  good  performers. 
The  Responsive  Configuration  is  viewed  as  an  ideal  type  in  a  political  economy  that 
requires  organizations  to  be  accountable  to  the  public  by  anticipating  and  responding  to 
its  concerns.  Responsiveness,  not  efficiency  or  effectiveness,  becomes  the  primary  driver 
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under  these  circumstances.  Whether  public  managers  view  there  roles  as  crisis  managers, 
who  strive  to  accommodate  political  interests  or  whether  they  see  themselves  as  power 
brokers  who  promote  their  preferred  interests,  responsiveness  becomes  the  mantel  by 
which  they  are  measured. 

4.  Adapter  Configuration 

The  Adapter  Configuration  seeks  to  optimize  organizational  effectiveness, 
downplaying  interest  in  efficiency.  Public  managers  achieve  organizational  efficiency  by 
adapting  to  the  external  environment  and  meeting  customer  needs.  They  decentralize 
decision  making  so  it  rests  on  the  shoulder  of  members  who  are  closest  to  customers, 
substituting  customers  for  bosses  as  the  basis  of  their  actions.  They  rely  on  the 
employee’s  up-to-date  knowledge  of  the  industry  and  customer  demands  to  provide 
insightful  and  innovative  ideas  to  meet  customer  expectations.  These  types  of 
organizations  reward  risk  takers  and  innovative  ideas.  Organizations  are  decentralized 
and  focus  on  collaborative  projects  and  cross-disciplinary  teams.  Employees  are  given  a 
general  vision  of  the  future,  rather  than  specific  goals  and  objectives,  in  order  to  inspire 
them  to  search  for  new  ideas  rather  than  a  planned  plotted  course.  Flexibility,  creativity, 
exploration  and  experimentation  are  far  more  important  to  managers  than  rigid  adherence 
to  internal  order  and  control.  Research  and  Development  organizations  are  typical  of 
those  in  the  Adaptive  Configuration.  National  Aeronautics  and  Space  Agency  (NASA) 
during  the  1960s  is  a  good  example  of  this  organizational  configuration.  Given  a  very 
general  mandate  by  President  Kennedy  to  land  a  man  on  the  moon  within  the  decade, 
NASA’s  goals  for  how  to  do  it  were  neither  fixed  nor  precisely  determined.  Addressing 
how  to  meet  this  mandate,  required  exploration,  experimentation  and  learning  from 
experience.  No  one  had  a  formula  for  what  had  to  be  accomplished.  NASA  developed 
an  agile  organization  where  innovation  would  be  more  important  than  stability.  As 
NASA’  administrator,  James  Webb’s  credo  was  “You  have  to  have  a  system  that  pennits, 
encourages,  and  even  forces  flexibility  and  adaptive  innovation  on  the  part  of  those 
people — including  the  executives — who  are  responsible  for  its  various  elements”.  Thus 
NASA  went  through  various  organizational  changes  to  ensure  flexibility  and  to  induce  a 
state  of  “organic  flux”.  The  efforts  produced  a  decentralized  system  reliant  on  program 
offices  and  field  center  for  project  management.  The  intention  of  senior  management 
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was  to  give  the  centers  what  they  needed  to  get  the  job  done  and  field  offices  were  given 
the  authority  to  act  independently  when  the  situation  warranted  it.  Although  managers 
were  expected  to  be  knowledgeable  in  sound  management  doctrine  and  practice,  they 
were  also  required  to  do  a  job  without  exact  definition  of  what  it  was  or  how  it  should  be 
done.  There  was  no  precedent  to  which  a  manager  could  examine.  He  was  on  his  own  to 
figure  out  a  new  or  different  solution.  Maneuverability  of  the  whole  and  the  varied  parts 
was  essential  for  success.  As  Webb  admitted,  “we  accepted  a  large  degree  of 
organizational  instability  in  order  to  achieve  the  necessary  degree  of  maneuverability.” 

5.  Generative  Configuration 

The  Generative  Configuration  demands  optimal  efficiency  and  effectiveness.  In 
this  configuration,  managers  strive  for  both  as  they  search  for  ways  to  reconcile  the 
competing  expectations  emanating  from  the  two  dimensions.  Central  to  operating  in  this 
type  of  configuration  is  the  reliance  on  stakeholder  collaboration.  This  is  the  first  time 
we  see  the  mention  of  stakeholder  collaboration  in  our  configuration  models.  Usually  we 
are  pitted  against  various  stakeholders  often  with  differing  interests  than  ours.  The 
generative  organization  relies  on  collaboration  with  the  stakeholders  to  achieve  both 
organizational  efficiency  and  effectiveness.  Efficiency  derives  from  the  networked 
stakeholders  working  together  in  the  pursuit  of  common  means.  Effectiveness  derives 
from  the  networked  stakeholders  working  together  in  the  pursuit  of  common  ends. 
Stakeholder  collaboration,  the  key  element  to  a  generative  organization,  takes  on  many 
forms,  including  multi-functional  teams,  partnerships,  joint  ventures,  and  alliances. 
Stakeholders  may  be  internal  or  external  to  the  organizations.  However,  collaborations 
usually  require  participants  to  come  together  quickly  to  exploit  some  opportunity  or  solve 
some  problems  that  each  acting  independently  could  not  address.  The  point  is  to  leverage 
intellect  and  promote  peoples  generative  learning — learning  that  develops  their  capacity 
to  create  new  solutions  to  old  problems  rather  than  settle  for  adaptive  learning  that  only 
prepares  them  for  coping.  The  intent  of  generative  learning  is  to  open  up  new  ways  at 
looking  at  the  world  and  to  encourage  deeper  understanding  of  a  system  and  its 
underlying  dynamics.  Finding  public  organizations  that  exemplify  all  the  characteristics 
of  the  Generative  Organization  proves  to  be  more  difficult  than  it  was  for  the  first  three 
configurations.  There  is  evidence  that  some  organizations  that  exemplify  some  traits  of  a 
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Generative  Organization,  but  on  a  whole  would  not  be  classified  as  generative.  There 
are:  stakeholder  dialogues  and  deliberations  to  craft  public  policy  and  organizational 
direction;  learning  strategies  to  review  organizational  procedures  and  chart  new  courses 
of  action;  measures  of  outputs  and  outcomes  to  track  and  improve  organizational 
performance;  dedications  to  customer  service  and  offering  only  those  products  and 
services  that  add  value.  While  lacking  a  true  holistic  generative  organization,  the  many 
aspects  that  define  a  learning  organization  exist  in  the  some  public  organizations  today. 

6.  Why  is  the  Configuration  Approach  Important  in  Determining  the 
Efficiency  and  Effectiveness  of  a  Field  Contracting  Office? 

In  Chapter  12  of  Nancy  Robert’s  book,  the  configuration  approach  to 
management  has  provoked  interest  in  the  part  of  practitioners  and  scholars,  and  support 
continues  to  grow  as  researchers  test  the  various  configurations  in  different  organizations 
and  environments.  The  four  configurations  simplify  the  management  jungle  of  competing 
theories  and  models  for  public  managers  and  at  the  same  time,  raise  the  following 
fundamental  questions:  How  efficient  are  we  and  how  efficient  must  we  become  to 
reduce  costs  and  conserve  resources?  How  effective  are  we  in  meeting  customer  needs  in 
a  changing  environment?  How  responsive  are  we  to  stakeholder  issues  and  how 
responsive  must  we  be,  given  competing  political  interests? 

In  taking  a  configuration  approach,  the  component  parts  of  each  ideal 
configuration  “fit”  together.  Fit  reveals  the  extent  to  which  organizational  components 
are  mutually  reinforcing  and  compatible.  When  the  components  fit  and  are  patterned 
after  a  configurations  ideal,  the  organization  is  expected  to  be  a  high  performer; 
deviations  from  ideal  types  do  not  perform  as  well.  This  is  an  important  concept  to 
understand,  especially  as  it  relates  to  change.  Managers  must  understand  how  the  change 
may  effect  the  “fit”  to  their  organization.  For  example,  wholesale  adoption  of  techniques 
because  they  are  promoted  as  “good  business  practice”  from  the  private  sector,  without 
consideration  of  organizational  fit,  risks  misaligning  the  organization  and  it  component 
parts,  which  runs  the  risk  of  reducing  rather  than  improving  organizational  performance. 

While  the  Generative  Configuration  is  evolving  and  has  not  been  found  in  its 
entirety  as  a  separate  configuration,  it  captures  an  important  element — the  network 

structure — that  has  been  missing  from  traditional  analysis  of  this  area.  Both  the 
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Responsive  and  the  Generative  configurations  capture  characteristics  particularly  relevant 
in  the  public  arena.  Each  acknowledges  the  importance  of  stakeholders  to  the 
organization’s  performance:  the  Responsive  Configuration  assumes  a  competitive  model 
of  stakeholder  relations,  while  the  Generative  Configuration  assumes  a  collaborative 
model  of  stakeholder  relations.  Two  of  the  ideal  types  (Directive  and  Adaptive)  have 
been  demonstrated  to  be  high  perfonners,  and  the  third  (Responsive)  a  good  performer  in 
a  regulated  environment.  Therefore,  we  find  there  are  multiple  paths  to  high 
organizational  perfonnance  and  “one  size  does  not  fit  all”  and  one  configuration  is  “not 
better  than  the  other”. 

High  performing  organizations  tend  to  position  themselves  as  ideal  types  at  either 
end  of  the  efficiency/effectiveness  continuum.  Other  organizations  have  been  found  at 
varying  points  along  either  axis.  There  are  many  hybrid  organizations  found  on  the 
configuration  model.  These  types  of  organizations  have  dual  focus;  it  seeks  the 
flexibility  inherent  in  the  Adaptive  Organization,  and  the  control  and  order  characteristic 
of  the  Directive  Configuration.  Many  public  organizations  combine  the  efficiency  of 
machine  bureaus  with  the  adaptations  and  innovations  that  professional  bring  from  their 
fields.  This  allows  organizations  to  create  subunits  that  specialize  in  effectiveness 
through  innovation  and  change  and  customer  building,  while  another  subunit  can  mine 
traditional  products  and  services  to  achieve  greater  efficiency.  Thus  the  overall  corporate 
entity  can  benefit  because  it  combines  subunits  striving  for  either  effectiveness  or 
efficiency. 

In  order  to  determine  the  effectiveness  and  efficiency  of  any  organization,  it  is 
imperative  that  you  first  detennine  where  you  find  yourself  on  the  organizational 
configuration  model.  After  you  have  done  this,  you  can  plot  a  course  of  action  and 
management  strategy  to  achieve  the  organizational  goals.  At  this  point  you  may  be  able 
to  offset  the  competing  interests  of  the  efficiency  vs.  effectiveness  paradox. 

That’s  why  it  is  imperative  that  contracting  offices  understand  how  they  are 
configured  organizationally  so  that  they  can  make  tradeoffs  on  efficiency  and 
effectiveness.  In  other  words,  why  strive  for  effectiveness  if  you  are  strictly  a  Directive 
Organization. 
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7. 


Where  Do  Most  Field  Contracting  Offices  Fit  in  the  Organizational 
Configuration  Model? 


HIGH 


EFFECTIVENESS 


LOW 


LOW 


HIGH 

EFFICIENCY 


Figure  2.  Field  Contracting  Officer  (FCO)  Organizational  Configuration 


The  researcher  took  an  in  forma  I  survey  as  well  as  examined  several  field 
contracting  offices  mission  statement  and  strategic  plans  to  detennine  where  they  fit  on 
the  Organizational  configuration.  As  depicted  in  Figure  2,  most  see  themselves  as  a 
hybrid  organization.  Continuously  striving  for  customer  satisfaction  (effectiveness)  and 
supporting  the  war-fighter  while  offsetting  that  with  ways  to  reduce  infrastructure  and 
cycle  time  (efficiency).  Most  Field  Contracting  Offices  find  themselves  being  graded  on 
how  effective  they  are  in  meeting  both  customer  needs  and  the  multiple  stakeholder 
interests.  When  asked  how  they  balance  efficiency  with  effectiveness  most  mentioned 
the  balanced  scorecard  as  an  approach  to  measuring  not  just  effectiveness  and  efficiency, 
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but  also  the  overall  performance  of  the  entire  organization  in  how  it  meets  its  mission  and 
supports  its  employees.  The  researcher  now  will  examine  the  balanced  scorecard 
approach  and  how  it  relates  to  the  organizational  configuration  model.  In  addition,  the 
researcher  will  provide  some  success  stories  of  the  balanced  scorecard  approach  and  how 
it  may  help  in  developing  an  organizational;  model  for  evaluating  efficiency  and 
effectiveness. 

D.  THE  BALANCED  SCORECARD  MANAGEMENT  SYSTEM 

In  John  Gray’s  thesis  (Measuring  Satisfaction  in  the  Program  Manager  - 
Procuring  Contracting  Officer  Relationship,  December  1997),  the  balanced  scorecard 
management  systems  allows  managers  to  look  at  their  business  from  four  important 
perspectives  (financial,  internal  business,  learning  and  growth,  and  customer).  The  four 
perspectives  of  the  scorecard  take  into  account  the  duality  of  short-  and  long-  term 
objectives,  desired  outcomes  and  perfonnance  constraints,  and  objective  and  subjective 
measures.  This  system  is  useful  because  it  recognizes  that  executives  do  not  rely  on  one 
performance  measure  to  manage  an  organization.  Instead  it  offers  managers  a  balanced 
presentation  of  both  financial  and  operational  matters. 

The  financial  perspective  included  performance  measures,  which  indicate  whether 
“the  organization’s  strategy,  implementation,  and  execution  are  contributing  to  bottom- 
line  improvement”  (Kaplan  &Norton,  1992,  p.77).  The  internal  business  perspective 
provided  “measures  which  focus  on  the  internal  processes  that  will  have  the  greatest 
impact  on  customer  satisfaction  and  achieving  an  organization’s  financial  objective” 
(Kaplan  &  Norton,  1996,  p.27).  The  learning  and  growth  perspective  identifies  the 
“infrastructure  that  the  organization  must  build  to  create  long-term  growth  and 
improvement.  This  is  the  rationale  for  significant  investment  in  reskilling  employees,  in 
information  technology  and  systems,  and  enhanced  organizational  procedures”  (Kaplan 
&  Norton,  1996,  p.  12).  The  customer  perspective  comprises  “generic  measures  of  the 
successful  outcomes  from  a  well  formulated  and  implemented  strategy.  The  core 
outcome  measures  include  customer  satisfaction,  customer  retention,  new  customer 
acquisition,  customer  profitability,  and  market  and  account  share  in  targeted  segments” 
(Kaplan  &  Norton,  1996,  p.26).  To  utilize  the  Balanced  Scorecard,  organizations  must 

first  establish  goals  for  each  of  the  perspectives,  and  then  translate  these  goals  into 
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specific  performance  measures.  (Kaplan  &  Norton,  1992)  The  major  advantage  of  the 
Balanced  Scorecard  management  system,  according  to  Litman  And  Wheeler  (1997)  is 
that  it  ensures  that  no  organizational  processes  are  ignored  and  that  all  types  of 
performance  measures  are  examined  by  organizational  decision-makers  to  present  a  clear 
picture  of  the  status  of  the  organization. 

The  Procurement  Task  Force  fonned  by  the  President’s  Management  Council  has 
recommended  that  agencies  utilize  the  Balanced  Scorecard  management  system.  Toward 
this  end,  the  Procurement  Task  Force  has  identified  four  major  goals  for  the  procurement 
system:  quality,  timeliness,  price  and  productivity,  which  organizations  can  utilize  to 
build  their  scorecard  systems.  Figure  3  represents  a  graphical  depiction  of  the  diamond 
shaped  balanced  scorecard  (www.isixsigma.com) 


Hovdo 
Customers 
see  us? 


Customer  Perspective 


GOALS 

tVEASURES 

Hovdo  ve  look  to 
our  shareholders? 


What  must 
ve  excel  at? 


Internal 

Business  Perspective 


Figure  3.  The  Balanced  Scorecard  (From  www.isixsigma.com) 


Many  changes  have  happened  to  the  balanced  scorecard  since  its  inception  in 
1992.  Although  the  concepts  behind  the  balanced  scorecard  are  fairly  simple  to 
understand,  they  are  difficult  to  implement  (Graham  Brown’s,  the  Winning  Score,  2000). 
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Graham  Brown  postulates  that  measurement  is  very  difficult  in  organizations  because  it  is 
not  an  exact  science  with  hard  rules  and  predictable  interrelationships  between  variables. 
People  run  organizations  and  their  actions  are  sometimes  inherently  unpredictable. 
Hence,  a  measure  that  made  perfect  sense  at  one  time  might  now  seem  laughable  when 
one  sees  the  behavior  it  drove  in  the  organization.  The  balanced  scorecard  is  all  about 
picking  the  right  metrics.  One  must  be  careful  about  how  they  select  metrics.  When  you 
focus  on  improving  one  metric,  two  other  metrics  might  decline.  We  find  samples  of  this 
in  the  decade  of  the  80s  and  90s.  The  80s  was  dedicated  to  improving  productivity,  as 
the  perception  was  that  the  work  force  was  lazy  and  unproductive.  Productivity 
improved  as  we  approached  the  close  of  the  decade,  but  quality  worsened.  The  decade  of 
the  90s  focused  on  the  quality  movement,  which  in  many  cases  lowered  financial  results. 
As  we  enter  the  21st  century,  the  new  mantra  for  organizational  performance  is  balance, 
hence  the  balance  scorecard  management  system. 

In  his  introduction,  Graham  Brown  extols  the  virtues  of  measuring  what  matters. 
He  cites  “if  you  find  that  the  chart  of  performance  data  you  have  been  examining  every 
month  for  the  past  year  is  merely  confirming  what  you  already  knew,  you  should 
consider  dropping  that  metric  from  your  scored  card”.  He  provides  the  following  tip 
concerning  measurement,  “Many  things  in  life  are  done  well  without  requiring  formal 
data.  Measure  only  those  factors  that  will  provide  you  with  infonnation  you  didn’t  know 
already  and  help  you  improve  performance  or  make  better  decisions”.  The  balanced 
scorecard  is  a  dynamic  performance  measurement  tool  and  has  helped  many 
organizations,  both  public  and  private,  but  it  requires  much  work  and  expertise  to 
properly  implement  it.  There  are  many  mistakes  made  in  determining  what  indeed  are 
appropriate  measures.  Graham  Brown  provides  a  list  of  the  top  10  measurement 
mistakes.  He  bases  this  on  his  years  of  consultation  with  both  public  and  private 
organizations: 

1.  Tracking  Output/Outcome  Metrics  that  cannot  be  influenced  or 
controlled. 

2.  Gathering  data  that  tells  you  what  you  already  know. 

3.  Gathering  data  for  its  own  sake. 

4.  Relying  heavily  on  Customer  Satisfaction  Surveys. 
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5.  Executives  focusing  on  detailed  metrics. 

6.  Measures  are  not  linked  to  the  strategic  plan. 

7.  Failing  to  define  practical  correlations  between  key  metrics. 

8.  Reporting  data  that  is  difficult  to  read  and  analyze. 

9.  “Superstitious”  process  metrics. 

10.  Measures  that  drive  the  wrong  performance. 

The  challenge  is  choosing  the  right  metric  for  the  right  reason  for  the  organization, 
keeping  in  mind  the  strategic  goals  and  objectives.  The  Procurement  Executive  Council 
(PEC)  recommends  using  a  cost  to  spend  model  as  a  way  to  maximize  the  efficiency  of 
the  procurement  office  relative  to  purchasing  cost  (PEC  2001).  Later  in  Chapter  III,  the 
researcher  will  analyze  the  different  types  of  metrics  currently  gathered  for  a  typical  field 
contracting  organization. 

An  important  and  difficult  decision  that  needs  to  be  made  right  off  the  bat  is 
where  to  begin  the  scorecard  project.  One  simple  answer  is  to  begin  at  the  top. 
Conceptually  this  makes  a  lot  of  sense  because  it  is  easier  to  develop  lower  level 
scorecards  that  link  together  if  the  metrics  for  the  CEO  have  been  defined  first  (Graham 
Brown,  2000  p.232).  Many  companies  and  Government  agencies  have  implemented  the 
scorecard  at  many  different  levels  of  the  organization.  The  point  to  be  made  is  that  it  is 
fine  to  start  the  Scorecard  initiative  at  any  level  of  the  organization.  Starting  at  the  top  is 
probably  the  easiest  in  some  ways,  but  politically  this  is  often  just  not  possible.  Often, 
the  best  way  to  sell  the  CEO  or  Commanding  Officer  on  the  concept  is  to  show  it 
working  well  in  key  parts  of  the  business  or  organization  (Graham  Brown,  2000  p.233). 

E.  CHAPTER  SUMMARY 

This  chapter  included  an  overview  of  a  field  contracting  office  and  provided  a 
basic  understanding  of  where  a  field  contracting  office  fit  in  the  overall  Navy  Contracting 
function.  Also,  it  included  a  review  of  the  Navy  Transfonnation  effort  that  and  how  that 
impacts  a  field  contracting  office.  The  main  thrust  of  the  NAVSUP  transformation  is  “to 
better  structure  and  align  the  organization  to  most  effectively  and  efficiently  deliver 
combat  capability  through  logistics”.  With  that  being  the  mantra,  it  is  imperative  for  a 
field  contracting  office  to  ensure  that  they  are  as  efficient  and  effective  as  they  can  be. 
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Next,  the  chapter  focused  on  the  literature  review  that  was  conducted  concerning 
efficiency  and  effectiveness.  Based  on  this  review,  the  researcher  found  many  examples 
of  the  importance  associated  with  obtaining  efficiency  and  effectiveness.  The  GPRA  was 
the  main  catalyst  for  focusing  Government  agencies  on  measuring  outcomes  and  results 
and  moving  away  from  just  measuring  outputs.  The  literature  review  included  an 
analysis  of  many  GAO  studies  as  well  as  endowment  studies  that  examined  how 
organizations  were  attempting  to  satisfy  the  requirements  of  the  GPRA. 

The  third  area  the  chapter  examined  was  the  organizational  configuration  model 
developed  by  Nancy  Roberts.  Robert’s  model,  which  describes  different  types  of 
organizations  (Directive,  Responsive,  Adaptive,  Generative),  gives  one  framework  to 
understand  where  a  field  contracting  office  fits  in  this  model.  By  understanding  the  “fit” 
of  an  organization,  you  can  better  strategize  how  to  achieve  the  balance  between 
efficiency  and  effectiveness. 

Finally,  the  chapter  examined  the  balanced  scorecard  approach,  which  many 
organizations  are  currently  using  to  balance  all  aspects  of  their  operations.  The  balanced 
scorecard  approach  gives  an  organization  a  picture  of  how  they  are  performing  as  an 
overall  entity  and  looks  at  all  parts  of  the  organization  (financial,  internal  operations, 
customers  and  employees).  The  researcher  found  that  many  organizations  use  the 
balance  scorecard  approach  to  maximize  both  efficiency  and  effectiveness. 
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III.  DATA  PRESENTATION  AND  ANALYSIS 


A.  NAVY  STUDY’S 

The  researcher  found  a  summary  of  a  series  of  studies  perfonned  by  the  Navy  to 
evaluate  the  overall  performance  of  a  Contracting  Office.  The  summary  was  provided  by 
a  consultant  contractor,  Booz-Allen  &  Hamilton  (BAH),  in  response  to  another  study 
they  are  conducting  for  the  Navy  titled,  “Establishing  the  Effectiveness  and  Efficiency  of 
the  Department  of  Navy’s  Material  Establishment”.  The  researcher  was  not  able  to 
obtain  a  copy  of  the  various  studies  cited  in  the  BAH  report,  however,  BAH’s  summaries 
provided  the  researcher  with  a  starting  point  to  analyze  how  the  Navy  has  historically 
looked  at  the  efficiency  and  effectiveness  of  the  Contracting  function.  The  studies  did 
not  limit  themselves  to  a  field  contracting  office,  in  fact,  most  of  the  studies  were 
performed  to  detennine  the  most  effective  way  to  organize  the  contracting  function  in  the 
Systems  Commands  or  the  Navy  overall.  Unfortunately  most  of  the  studies  to  date  draw 
on  anecdotal  evidence  and  therefore,  they  only  generally  address  and  hypothesize  on  the 
efficiency  and  effectiveness  of  a  contracting  office.  However,  they  did  address  areas 
concerning  contracting  office  roles  and  responsibilities  as  well  as  overall  efficiencies  that 
can  be  gained  by  consolidating  offices.  There  are  some  aspects  to  the  studies  that  did 
address  cost  and  effectiveness  of  operations.  Efficiency  was  addressed  on  an  enterprise¬ 
wide  basis.  The  studies  provide  a  background  into  what  the  Navy  leadership  considers 
relevant  when  assessing  the  effectiveness  of  its  contracting  function.  As  you  will  see  in 
the  analysis  of  the  studies,  there  still  seems  to  remain  a  perception  that  all  contracting 
professional  (also  known  as  “1102s”)  are  created  equally  with  the  same  skill  mix  to 
acquire  any  type  of  supply  or  service. 

In  addition,  the  researcher  found  a  private  sector  study,  sponsored  by  the  National 
Association  of  Purchasing  Managers  (NAPM),  that  analyzed  what  measures  are 
considered  useful  to  a  CEO  when  assessing  the  overall  effectiveness  of  his  purchasing 
operations.  Also,  another  private  study,  sponsored  by  NAPM,  analyzed  the  roles  and 
responsibilities  of  Purchasing  Office  and  how  it  relates  to  the  overall  strategy  of  the 
company. 
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The  remaining  portion  of  this  chapter  will  show  the  name  of  the  study,  purpose, 
and  the  Analysis/Metric  Used.  Then  the  findings  or  recommendations  will  be  provided 
with  an  overall  analysis  of  the  study.  The  researcher  will  then  analyze  NAVSUP  and 
FISC  Norfolk’s  current  metrics  and  review  what  the  Defense  Contract  Management 
Agency  (DCMA)  measures. 


NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC 

USED 

NAVSEA  02  HQ 

12/17/91 

To  consider  the  consolidation  of 

1.  Consolidate 

Contracting 

contracting  organization 

three  alternatives: 

Consolidation  Study 

functions. 

-  Status  Quo 

Is  consolidation  feasible? 

-  Naval  Systems 

Contracting  Command 

Could  it  continue  to  provide 

with  HCA  authority,  with 

adequate  support  to  customers? 

contracts  personnel  co¬ 
located  with  customers. 

Would  it  result  in  efficiency 

-  Combine  Hardware 

savings  through  personnel 

Systems  Commands  into 

reductions? 

one  SYSCOM  with  the 
resulting  contract 
organization  moved  to 
one  site 

2.  Considered  pros 

and  cons  of  each 
alternative  based  on 
anecdotal  evidence. 

3.  Data  used  was 

primarily  personnel 
headcount,  and  projected 
reductions  that  were 
anecdotal. 

Table  1  NAVSEA  02  HQ  contracting  Consolidation  Study 

1.  Recommendations  and  Findings 

1.  Recommended  maintaining  the  status  quo. 

2.  Efficiencies  (personnel  reductions  gained  only  through  consolidation  of 
support  and  front  office  functions. 

3.  Unclear  that  any  efficiency  savings  were  gained  in  the  buying  function 
(number  of  touch  personnel  assumed  to  be  consistent  with  workload). 
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2.  Analysis 

While  this  study  provides  a  historical  contracting  perspective  of  consolidation  that 
spans  back  to  1991,  it  lacks  actual  data  analysis  and  relies  on  anecdotal  evidence. 
Finding  3  cited  above  gives  the  impression  that  the  number  of  people  is  assumed  to  be 
consistent  with  workload.  This  is  not  always  true  in  the  contracting  organizations 
interviewed  as  well  as  what  has  historically  transpired  at  the  researcher’s  contracting 
activity.  In  addition,  the  study  does  not  define  what  workload  actually  means.  Is  it  the 
number  of  purchase  requisition  on  your  desk?  The  number  of  delivery  orders  processed? 
Or  does  it  include  the  unmeasured  workload  of  acquisition  planning  and  market  research? 
The  different  complexities  of  the  workload  must  be  examined  in  order  to  make  the 
statement  that  the  number  of  personnel  is  consistent  with  the  workload.  One  complex 
service  acquisition  could  be  considered  equivalent  to  100  delivery  orders  as  they  relate  to 
personnel  and  workload. 


NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC 

USED 

ASN  RD&A  (ABM) 

6/28/95 

Provide  assessment  of  how 

Considered  three  alternatives: 

Study  on  Contracting 

contracting  could  be  potentially 

Functional  input 

organized  under  a  single 

1 .  Status  Quo 

SYSCOM,  including: 

2.  Examine  a  CEO 

-  Role  of  Contracting  in  the  Navy 

Organization.  (Locates 
functional  communities 

-  Assumptions  and  precepts  of 

with  customers  as  well  as 

successful  contracting  function. 

along  product  lines  in  an 

IPT  format) 

-  Recommend  an  organizational 
structure  (as  well  as  analysis  of 
alternatives  considered) 

3 .  Examined  a  Matrix 
organization,  which  is 

-  Investigate  cultural  norm  of  the 

similar  to  a  CEO  but 

contracting  community. 

without  a  Board  of 

Directors  or  product  line 
affiliation. 

.  Considered  pros  and  cons  of 
each  alternative  based  on 
anecdotal  evidence. 

Table  2  ASN  RD&A  Report  on  Contracting  Functional  Input 
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3.  Findings/Recommendations 


1.  Contracting  personnel  evaluations  must  remain  “stovepiped”  to  maintain 
independence. 

2.  Current  contracting  organizations: 

a.  Accommodates  geographical  dispersions  easily. 

b.  Gets  the  job  done. 

c.  Presents  limited  opportunity  for  consolidation  of  redundant 
activities  (policy  and  management  groups  are  in  most  activities). 

d.  Does  not  facilitate  inter-SYSCOM  or  inter-Command  interaction. 

3.  Recommended  CEO  Organization  because  it  rationalizes  redundant  work 
across  material  establishment  and  provides  a  single  point  of  contact  for  all 
acquisition  matters. 

The  following  other  alternatives  were  noted,  but  not  considered,  including: 

Centralization  at  one  site.  Not  considered  because  of  degradation  in 
service  to  customer. 

Single  SYSCOM  with  regional  offices.  Not  considered  because  it  creates 
redundant  functions. 

Product  oriented  contracting  office.  Not  considered  because  it  was  too 
similar  to  CEO  and  matrix. 

4.  Analysis 

This  study  presented  a  few  interesting  ideas  on  restructuring  contracting  on  an 

enterprise-wide  basis.  However,  once  again  it  was  based  on  anecdotal  evidence  with  no 

real  investigation  of  contracting  office  efficiencies  by  combining  functions  or 

products/services.  All  the  analysis  was  based  on  a  theoretical  framework.  This  study 

begs  the  follow-up  question  of:  Were  there  counterpart  studies  to  this  analysis  to  include 

consolidation  of  the  financial  or  inventory  function?  In  addition,  the  study  recommended 

implementation  of  the  CEO  organization,  yet  the  Navy  for  the  most  part  completely 

ignored  this  alternative.  It  is  interesting  to  note  that  one  of  the  findings  was  that 

“contracting  personnel  evaluations  must  remain  “stovepiped”  to  maintain  independence”. 

The  study  then  recommends  the  “CEO  organization”  as  a  way  of  combating  this 

theoretical  inefficiency.  Much  has  changed  since  this  study  was  performed  in  1995.  The 

advent  of  the  Integrated  Product  Team  (IPT)  has  changed  the  way  the  organization  feels 

about  the  stove-piped  nature  of  the  contracting  function.  All  the  stakeholders  in  an  IPT 
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bring  a  functional  expertise  to  the  team  that  allows  early  identification  of  problems  and 
issues  and  considers  all  functional  areas  (financial,  legal,  contracting,  engineering,  and 
logistics)  integral  to  the  success  of  the  acquisition.  So  to  some  degree  in  many 
organizations,  the  CEO  philosophy  has  taken  hold.  However,  in  my  interviews  with 
other  field  contracting  offices,  IPTs  are  rarely  officially  formed  because  the  participants 


involved  are  from  different  customer  activities. 


NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC 

USED 

ASN  RD&A  (ABM) 

1/22/98 

Baseline  current  organizational 

Considered  the  following 

Navy  Contracting 

structure. 

alternatives: 

Organizational 
Alignment  Study 

Assess  organizational 

1.  Transfer  some  contracting 

Team  (NCOAST)) 

alternatives,  including  evaluation 

resources  to  fleet  but 

of: 

maintain  current  HCA 
placement. 

-  Top  level  costs 

-  Responsiveness  to  customer 

2.  Transfer  some  contracting 

-  Advantages  and 

resources  to  fleet  and 

Disadvantages 

centralize  HCA  placement. 

Conduct  analysis  around  the 

3.  Transfer  some  contracting 

hypothesis  that  “smaller 

resources  to  fleet  and 

organizations  could  be 
consolidated  with  larger 

delegate  HCA  to  fleet. 

contracting  organizations  to  yield 

4.  Transfer  some  contracting 

labor  dollar  savings”. 

resources  to  fleet  and 
delegate  HCA  to  regional 

Conduct  regional  analysis  of 
consolidation  in  Hampton  Roads 

commanders. 

and  Northwest  Region  of  the 

5.  Consolidate  all 

United  States. 

contracting  functions  into  a 
single  SYSCOM. 

Survey  contract  customers  to 
determine  current  satisfaction 

6.  Consolidate  field 
activities  into  regional 
commands. 

7.  Collected  data  through 
comprehensive  data  call  to 
over  600  organizations  and 
queried  contracting 
customers  via  surveys 

Used  the  following  evaluation 
criteria  to  evaluate  alternatives: 
Responsiveness,  Cost, 
Accountability,  and  Regions 

Table  3  ASN  RD&A  Navy  Contracting  Organizational  A] 

lignment  Study  Team 

(NCOAST) 
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5. 


Findings/Recommendations 


The  baseline  organization  consisted  of  570  activities,  10,000  personnel,  1.3M 
actions,  and  $39B  in  obligations  and  $500M  in  direct  labor  cost.  The  study  found  that 
there  was  an  organic  cost  savings  potential  of  $83  to  $199M  (16-39%  savings)  by 
consolidating  activities.  The  study  also  indicated  that  workload  was  declining,  which 
would  be  expected  given  the  reduction  in  the  Defense  budget. 

The  Study  observed  the  following  about  the  status  quo  structure  of  the  contracting 
function: 

High  customer  satisfaction  due  to  decentralization  and  dedicated 
resources. 

Inhibits  sharing  of  best  practices  within  and  among  HCA’s. 

Responsibility  and  accountability  are  not  consistently  placed  with 
organizations  responsible  for  the  mission. 

There  is  substantial  duplication  of  effort. 

Contracting  missions  are  misaligned  (ownership  and  management 
of  contracting  function  is  inconsistent  with  mission). 

The  study  also  noted  the  following  challenges  associated  with  consolidation  or 
regionalization: 

Geographical  proximity  to  customer. 

Alignment  of  ownership  and  management  of  contracting  resources. 
Productivity  and  workload,  cost,  customer  satisfaction,  and 
accounting  difficulties  between  an  Operations  and  Maintenance 
Navy  (O&MN)  and  Navy  Working  Capital  Funds  (NWCF) 
activities. 

The  study  found  that  no  matter  which  organization  structure  selected,  significant 
cost  savings  could  be  achieved  by  eliminating  small  activities  and  they  cited  a  1992  CNO 
study).  They  also  found  that  customers  considered  the  following  important  when 
discussing  contracting  personnel: 
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Geographical  proximity. 

Centralized  contracts  for  supplies. 

Dedicated  contracting  support. 

Use  of  Purchase  Cards. 

While  the  study  was  probably  the  most  comprehensive  analysis  of  the  contracting 
function  navy-wide,  it  only  recommended  a  few  courses  of  action. 

1.  Reduce  the  amount  of  activities  with  contracting  authority.  This 
will  allow  for  infrastructure  reduction  and  regionalization.  This 
may  also  lead  to  increasing  customer  responsiveness  by 
restructuring  to  better  align  resources  to  support  mission  functions. 

2.  Increase  Purchase  Card  authority  to  $  100K. 

3.  Conduct  further  analysis  of  consolidation  of  contracting  operations 
in  the  Hampton  Roads  area. 

4.  Establish  a  navy-wide  database  to  track  direct  labor  costs, 
overhead,  actions,  and  obligations  and  other  contracting  functions 
to  quantify  efficiency  and  effectiveness  and  request  contractor 
support  to  do  further  analysis  to  look  at  diversity  of  workload  and 
its  impact  on  customer  satisfaction. 

5.  Accelerate  full  implementation  of  SPS  in  order  to  increase 
efficiency  and  permit  record/document  sharing  between  activities. 

6.  Analysis 

The  NCOAST  study  was  probably  one  of  the  most  comprehensive  reviews  of 
Contracting  functions  in  Navy  organizations.  It  consisted  of  data  analysis  that  included 
actions,  dollars,  direct  labor  costs,  indirect  costs,  mission  funded  organizations  and  Navy 
working  capital  fund  organizations.  The  study  also  provided  HCA  responsibilities,  along 
with  source  of  responsibility  and  ability  to  delegate  decision-making  to  lower  levels. 
Some  of  the  findings  were  surprising  (centralized  contracts  for  supplies).  It  is  hard  to 
understand  why  a  customer  would  really  care  if  the  contracting  office  centralized 
contracts  for  suppliers.  One  would  think  that  a  customer’s  main  interest  is  getting  what 
they  need  when  they  need  it  at  reasonable  prices.  And  others  were  obvious  (reducing  the 
amount  of  offices  with  contracting  authority  will  reduce  infrastructure  costs).  This  action 
in  and  of  itself  will  reduce  infrastructure  but  it  may  increase  costs  in  other  parts  of  the 
enterprise  and  that’s  why  a  thorough  efficiency  review  or  business  case  analysis  must  be 
performed  before  implementation  of  any  action.  While  the  study  was  comprehensive  in 
its  look  at  cost  associated  with  the  contracting  function,  it  failed  to  look  at  complexity  of 
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workload,  and  the  system’s  ability  to  consolidate  work  in  a  center  of  expertise  concept.  In 
addition,  the  recommendation  to  establish  a  navy-wide  database  to  collect  important 
contracting  data  seemed  to  be  a  reasonable  recommendation  that  would  gather  real  data, 
while  providing  a  useful  tool  to  help  manage  the  Navy’s  contracting  community. 
Unfortunately,  to  the  best  of  the  researcher’s  knowledge,  this  recommendation  has  not 
been  implemented  and  the  Navy  to  date  still  has  not  conducted  an  analysis  of  workload 
complexity  and  how  it  relates  to  improving  overall  efficiency  and  effectiveness. 

The  NCOAST  charter  was  to  conduct  an  evaluation  of  existing  contracting 
organizational  structures,  relationships  and  processes  to  identity  alternatives  that  will 
enable  the  Navy  to  better  support  its  customers  (effectiveness),  streamline  processes 
(efficiency),  reduce  infrastructure  (effectiveness)  and  eliminated  inefficiencies 
(efficiency).  The  NCOAST  team  went  to  great  lengths  to  analyze  customer  support  and 
reduce  infrastructure  goals,  but  did  not  dedicate  much  effort  in  streamlining  policies  and 
eliminating  inefficiencies.  The  significance  of  this  study  was  its  reliance  on  actual  data 
and  not  anecdotal  analysis.  However,  it  also  relied  on  much  anecdotal  evidence.  For 
instance,  a  hypothesis  was  developed  proposing  that  smaller  contracting  organizations 
could  be  consolidated  with  larger  contracting  organizations  yielding  labor  dollar  savings. 
A  methodology  was  developed  (based  on  this  hypothesis)  to  mathematically  represent 
labor  dollar  savings  within  the  Navy  contracting  organizational  structure.  This 
methodology  was  based  on  a  Center  for  Naval  Analysis  (CNA)  study  that  theorized  20- 
50%  labor  savings  based  on  consolidating  existing  organizations.  The  team  used  labor 
dollars  from  the  Hampton  Roads  and  Northwest  regions  to  build  a  Navy-wide  model. 
This  hypothesis  for  projecting  navy-wide  labor  dollar  savings  using  the  Hampton  Roads 
and  northwest  region  as  a  benchmark  appear  misleading  as  the  majority  of  smaller 
activities  are  located  in  San  Diego  and  Hampton  Roads/Washington  DC  area.  In 
addition,  CNA  only  theorized  a  20-50%  savings,  which  first,  is  a  large  percentage 
difference  and  second,  is  not  developed  on  factual  data,  just  theorized  savings. 

The  NCOAST  study  did  take  a  significant  look  at  customer  satisfaction.  This 
measure  of  effectiveness  seemed  to  weigh  heavily  in  their  recommendations  as  presented 
above.  While  the  data  analysis  did  not  go  deep  enough  on  the  efficiency  or  productivity 
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on  contracting  offices,  it  did  provide  enough  background  to  allow  further  analysis  of  the 
contracting  function. 


NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC  USED 

ASN  RD&A  (ABM) 

The  Hunt  for  10%: 

A  Framework  for 
Attacking  Contract 
Costs 

3/25/99 

Reduce  contract  cost  to  the 

Navy  by  at  least  10%  and 
deliver  the  same  quality 
product. 

Put  a  detailed  global  strategic 
plan  in  place 

Identify  those  areas  in  industry 
where  this  pressure  would 
potentially  derive  the  most 
savings. 

Study  identified  savings  opportunities  that 
involved  consolidation  at  3  levels: 

-  Consolidate  common  requirements 
across  HCA’s. 

-  Consolidate  common  contract 
support  functions. 

-  Consolidate  within  HCA 

The  process  for  evaluating  and  arriving  at 
savings  opportunities  was  not  outlined. 

The  contracting  markets  in  which  the 

Navy  participates  and  approximate  dollar 
amounts  spent  (FY  98)  were  identified 
along  with  the  number  of  Navy 
contracting  activities  by  region. 

Table  4  ASN  RD&A  (ABM)  The  Hunt  for  10%:  A  Framework  for  Attacking  Contract 

Costs 

7.  Findings/Recommendations 

The  study  identified  a  decrease  in  dollars  spent  on  supplies  and  research  and 
development  contracts  from  1988-1997,  while  service  contract  Dollars  remained  steady. 
In  addition,  the  service  to  supply  ratio  more  than  doubled  from  1988  -  1997  (i.e.,  service 
contracts  went  from  24%  of  contract  dollars  in  1988  to  40%  in  1997,  while  supplies  went 
from  58%  to  42%  in  the  same  period). 

8.  Analysis 

This  study  was  perfonned  as  a  preview  to  the  next  study.  The  working  group 
provided  a  basic  framework  for  the  next  team  to  further  analyze  potential  savings  to  the 
Navy.  The  next  study  team  reported  out  25  days  after  this  working  group  and  their  study 
is  provided  for  review  next. 
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NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC 

USED 

ASN  RD&A  (ABM) 

Red  Ink  Rising:  A 
Framework  for 
Attacking  Contract 
Costs 

4/19/99 

Reduce  contract  cost  to  the 
Navy  by  at  least  10%  and  deliver 
the  same  quality  product. 

Put  a  detailed  global  strategic 
plan  in  place 

Identify  those  areas  in 
industry  where  this  pressure 
would  potentially  derive  the 
most  savings. 

Analysis  identifies  and  defines 
key  characteristics  for  each  market 
including  commercial 
applicability,  economies  of  scale, 
proprietary  data,  entry/exit  barriers 
and  legal  barriers. 

Market  share  by  industry 
classification  analysis  identifies  the 
Navy’s  share  of  spend  (by  dollar 
amount  and  percentage)  in  each 
industry/market  it  participates. 

Study  relied  on  1998  data. 

Table  5  ASN  RD&A  (ABM)  Red  Ink  Rising:  A  Framework  for  Attacking  Contract  Costs 


9.  Findings/Recommendations 

The  overall  findings  of  the  working  group  were  as  follows: 

The  Navy  interacts  in  2  market  cells  within  a  3  X  3  customer/seller 
matrix,  few  to  few  and  many  to  many. 

An  example  of  few  to  few  is  Research  &  Development  (aircraft  & 
airframes,  ships,  engines/turbines. 

An  example  of  many  to  many  is  maintenance  and  repair  of 
equipment,  professional  and  management  services,  maintenance  of 
real  property,  ADP  and  telecommunication  services. 

There  is  no  low  hanging  fruit  or  obvious  opportunities  for 
leveraging.  Sellers  are  decreasing  and  future  contract  costs  are 
being  driven  by  current  acquisition  decisions.  The  group  found 
that  the  ultimate  responsibility  for  savings  is  at  the  working  level. 


The  study  group  recommended  that  the  Navy  institute  market  sector  brokers  to 
facilitate  contracting  strategy  development  for  the  procurement  of  common  and  related 
requirements  at  the  appropriate  level  within  the  “corporate”  Navy.  Also,  the  team 
recommended  that  the  Navy  team  with  contractors  to  pursue  reduced  infrastructure  and 
indirect  cost  rates  at  prime  and  subcontract  levels  and  the  navy  should  also  pursue 

improved  cash  management  of  unexpended  contract  funds. 
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10.  Analysis 

The  idea  of  having  a  “Corporate  Strategy”  for  the  contracting  and  requirements 
function  is  a  novel  idea.  By  establishing  market  sector  brokers  or  market  managers,  the 
Navy  can  begin  to  consolidate  their  requirements  and  allow  the  Contracting  Office  with 
the  most  expertise  in  that  market  to  satisfy  the  requirement.  In  theory,  we  will  have  more 
leverage  in  dealing  with  the  private  sector  and  in  addition,  contracting  offices  can  become 
much  more  efficient.  By  establishing  “Centers  of  Excellence”  for  contracting  that  utilize 
the  specific  expertise  of  that  contracting  office,  the  Navy  can  ultimately  reduce 
infrastructure  costs  across  the  Navy  enterprise  via  either  strategic  sourcing  or  functional 
assessments  to  detennine  “expert”  offices.  This  would  also  change  buying  habits,  hold 
contract  cost  steady  and  provide  leverage  to  private  industry  solutions.  Unfortunately, 
there  are  no  data  sources  identified  in  the  study,  so  the  researcher  can’t  determine  where 
the  study  group  received  their  information. 


NAME 

DATE 

PURPOSE 

ANALYSIS/METRIC 

USED 

Center  for  Navy 

3/09/00 

Assess  potential  for  cross¬ 

Analysis  included  an 

analysis  Study 

command  contracting 

evaluation  of  FY  99  obligations 

Briefing 

across  product  lines  and 

Recommend  approach  for 

SYSCOMS. 

Potential  Savings  from 

cross-command  contracting 

cross  command 

Analysis  also  included  a 

contracting 

Identify  other  approaches  for 

description  of  ongoing  cost 

reducing  contract  costs. 

reduction  initiatives 

Table  6  Center  for  Navy  Analysis  Study  Briefing:  Potential  Savings  From  Cross- 

Command  Contracting 

11.  Findings/Recommendations: 

The  overall  findings  of  the  working  group  were  as  follows: 

There  is  an  overall  lack  of  requirements  visibility  before  award. 

There  is  a  lack  of  Cross-Command  initiatives. 
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There  are  numerous  initiatives  underway,  however,  most  are 
internal  to  Commands. 

Lack  of  command-level  incentives  to  pursue  savings  initiatives. 

Data  are  not  available  to  maximize  opportunity.  Either  the 
information  is  not  available  within  or  among  activities  or  the  data 
systems  are  only  good  after  the  fact. 

The  study  group  did  not  making  any  recommendations.  However,  they  did 
provide  an  outline  of  “models  of  consolidation”  for  industry.  Unfortunately,  the 
researcher  could  not  get  a  copy  of  this  model  and  to  the  best  of  his  knowledge;  this  model 
has  not  been  piloted  or  implemented  at  any  Navy  Command. 

12.  Analysis 

While  the  study  indicated  that  FY  99  data  was  used,  it  does  not  indicate  the  data 
source.  It  is  difficult  to  determine  exact  product  lines  based  on  the  current  management 
information  systems  in  the  Navy.  To  the  best  of  the  researcher’s  knowledge,  most 
analysis  of  product  lines  or  services  is  done  via  a  study  of  the  Federal  Procurement  Data 
System  (FPDS).  The  FPDS  is  limited  in  that  it  only  categorizes  a  product  in  a  general 
Federal  Supply  Code  (FSC).  Without  actually  looking  at  the  contract,  it  is  difficult  to 
determine  if  the  supply  or  service  is  a  candidate  for  consolidation.  In  addition,  this 
analysis  is  all  after  the  fact  and  provides  us  little  benefit  in  forecasting  future 
requirements  or  even  forecasting  consolidated  requirements.  More  robust  management 
information  systems  need  to  be  implemented  that  tracks  the  requirement  form  initiation  to 
completion.  The  system  must  be  standardized  so  that  all  members  of  the  acquisition 
community  share  a  common  definition  of  the  product  or  service  being  acquired. 

On  October  24,  2002,  ASN  RD&A  (ABM)  issued  an  executive  direction  to  all 
Navy  Systems  Commands  addressing  the  results  of  an  Overhead  study.  The  researcher 
found  no  evidence  of  who  performed  the  study  or  what  data  they  used  in  performing  the 
study.  The  ASN  directive  inferred  that  a  consolidation  of  overhead  functions  could  save 
10%  or  roughly  60  Full  Time  Equivalents  (FTEs)  across  the  enterprise.  The  ASN 
directive  provided  a  detailed  look  at  Overhead  Activities.  The  researcher  recalls 
answering  a  data  call  in  2002  to  NAVSUP  concerning  FTEs  perfonning  overhead  duties. 
In  theory,  the  consolidation  of  overhead  type  functions  across  the  enterprise,  makes 
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sense,  after  all,  why  do  you  need  10  policy  directors.  Historically  we  organize  our 
activities  in  relationship  to  headquarters  and  there  is  a  trickle  down  effect  that  is 
continuously  happening.  For  example,  ASN  will  issue  a  policy  directive  to  all  the 
Systems  Commands,  the  Systems  Commands  will  then  put  their  spin  on  it  (which  is 
usually  just  reiterating  what  ASN  has  said)  then  the  Field  Contracting  Office  puts  their 
spin  on  it  and  implements  the  policy  or  guidance.  The  enterprise  can  reduce  their 
overheads  by  standardizing  policy  in  one  field  office,  whose  responsibility  it  is  for 
implementing  throughout  the  entire  enterprise,  thereby  eliminating  redundancy.  That  is 
just  one  example  of  savings  that  can  be  achieved  through  consolidation.  However,  for 
success  to  be  achieved  standards  have  to  be  established  and  buy-in  must  be  achieved 
across  the  enterprise.  Without  that  each  office  will  resort  back  to  old  ways  of  doing 
business  as  they  perceive  no  value  added  in  the  revised  structure.  Without  having 
knowledge  of  the  actual  study  that  ASN  performed,  the  researcher  will  assume  that  actual 
data  was  analyzed  in  order  to  come  up  with  the  60  FTE  savings.  The  only  concern  the 
researcher  has  is  in  ASN’s  definition  of  overhead  activities.  There  are  some  overhead 
activities  that  are  difficult  to  consolidate  among  offices,  like  bid  offices,  distribution  and 
file  room  maintenance. 

B.  A  PRIVATE  INDUSTRY  STUDY  ON  MEASURES  OF  PURCHASING 

EFFECTIVENESS 

In  1997,  Harold  Fearon  Ph.D.,  C.P.M  and  Bill  Bales,  C.P.M  conducted  a  research 
project  for  the  Center  for  Advanced  Purchasing  Studies  (CAPS  www.capsresearch.org) 
on  the  measures  of  purchasing  effectiveness.  The  foreword  of  their  study  state 
“Measuring  performance  of  the  purchasing  function  continues  to  be  a  vexing  problem. 
Hundreds  of  articles  and  monographs  have  been  written  on  the  subject  over  the  past 
several  years,  often  looking  for  a  "magic  formula."  If  there  is  one,  it  might  include  these 
steps: 

1.  Review  the  objectives  of  the  organization  to  determine  the 
measurements  needed. 

2.  Search  for  measures  that  match  with  the  goals. 

3.  Review  with  senior  management,  the  chief  purchasing  officer’s 
(CPO’s)  superior,  and  major  internal  customers  to  determine  what 
they  need. 
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4. 


Don’t  over  or  under  measure. 


5.  Recognize  that  what  you  measure  today  will  differ  from  what  will 
be  needed  tomorrow. 

The  objectives  of  the  study  were  to  determine  what  measures  are  considered  to  be  useful 
by  Chief  Purchasing  Officers  (CPOs),  and  Chief  Executive  Officers  (CEOs),  that 
measure  are  CPOs  currently  using  and  are  there  any  established  guidelines  CPOs  can  use 
to  select  purchasing  performance  measures. 

1.  Research  Findings 

There  was  considerable  variation  in  how  CPOs  viewed  the  purchasing  operations, 
and  in  many  cases,  it  all  depended  on  the  authority  delegated  to  them  by  leadership.  The 
research  showed  that  in  terms  of  what  the  CPOs  considered  to  be  the  most  valuable 
measures  for  assessing  purchasing  effectiveness  was: 

1 .  Use  of  leverage  through  combining  volumes. 

2.  Accurate,  timely,  and  efficient  data  collection. 

3.  Price  negotiations  resulting  in  savings. 

But  in  terms  of  actual  measure  usage,  the  three  highest-rated  measures  were: 

1 .  Price  negotiations  resulting  in  savings. 

2.  Department  budget  versus  actual  expenditure. 

3.  Past-delivery  performance. 

The  measures  in  the  organization,  managing  supplier  base,  and  overall  performance 
evaluation  areas  simply  are  not  receiving  the  same  relative  attention  as  those  in 
developing  supplier  base,  cost  effectiveness,  and  systems  utilization.  Also,  the  CPO 
found  that  the  five  least  important  measures  were: 

1 .  Percent  of  annual  dollar  volume  delivered  directly  from  supplier  to 
user. 

2.  Number  of  major  problems  solved  within  90  days. 

3.  Percent  of  suppliers  empowered  to  input  to  design  and 
specifications. 

4.  Purchasing  dollars  moved  from  outsourcing  to  in-house. 

5.  Number  of  supplier  satisfaction  surveys  this  year  versus  goal. 
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The  CPOs’  ranking  of  measurement  guidelines  showed  these  four  as  the  most  important: 


1 .  Chief  purchasing  officers  (CPOs)  must  constantly  be  on  the  alert  to 
add  new  and  appropriate  measurements,  and  to  delete  unneeded 
ones. 

2.  Measurements  are  primarily  for  the  purchasing  manager’s  use  in 
managing  and  monitoring  the  department. 

3.  Good  computer  systems  are  essential  in  supporting  the  flexibility 
needed  to  change  and  adapt  measurements. 

4.  Measurements  should  include  a  trend  analysis  to  indicate  direction 
of  progress. 

The  CEOs/presidents,  when  surveyed  on  their  need  for  purchasing  measures,  said  the 
most-needed  measures  primarily  were  non-financial  in  nature.  The  five  most-needed 
measures,  in  order,  were: 

1 .  Quality  of  purchased  items. 

2.  Key  supplier  problems  that  could  affect  supply. 

3.  Supplier  delivery  performance. 

4.  Internal  customer  satisfaction. 

5.  Purchase  inventory  dollars. 

The  five  least-needed  measures,  in  order  with  the  least-needed  shown  last,  were: 

1.  Purchases  from  minority-/female-owned  firms. 

2.  Purchasing  department  headcount. 

3.  Actual  price  paid  versus  a  published  market  basket  index. 

4.  Supplier  assessment  of  purchasing  perfonnance. 

5.  Purchasing  administrative  cost  as  percent  of  purchase  dollars 
spent. 

The  CEOs  wanted  a  trend  analysis  on  many  of  the  measures,  and  the  timing  of  their 
need  for  many  of  the  measures  varied.  Purchasing  should  check  with  senior  management 
to  see  what  specific  measures  are  needed,  how  frequently  the  information  is  needed,  and 
in  what  form  the  infonnation  should  be  presented. 
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2. 


Conclusions 


There  are  no  "quick  fixes"  to  the  purchasing  measurement  problem.  Each 
organization  is  unique  and  requires  measures  tailored  to  its  current  environment  and 
people.  However,  the  research  recommended  12  guidelines  that  should  be  followed  in 
establishing  a  measurement  system: 

1 .  Measures  need  to  be  designed  for  use  at  a  point  in  time. 

2.  Each  organization  has  specific  measurement  needs  at  a  given  point 
in  time. 

3.  Measures  should  address  financial  results,  supplier  performance, 
computer  systems,  and  internal  practices  and  policies. 

4.  Measures  must  change  frequently. 

5.  Trend  analysis  often  is  useful. 

6.  Measures  should  not  be  overdone  or  underutilized. 

7.  Measures  are  only  tools. 

8.  Benchmarking  is  a  source  of  new  ideas  and  measures. 

9.  Senior  management  must  see  value  in  the  measures  used. 

10.  Measures  can  show  the  effectiveness  of  purchasing,  and  identify 
areas  needing  improvement. 

1 1 .  Ensure  the  credibility  of  measures. 

12.  Continuous  improvement  in  purchasing  depends  on  measurement. 

3.  Analysis 

This  study  showed  some  significant  dichotomies  between  what  a  Chief 
Purchasing  Office  considers  important  to  the  effectiveness  of  a  purchasing  office, 
compared  to  what  a  Chief  Executive  Officer  finds  relevant.  CEOs  are  interested  more  in 
those  measures,  which  focus  on  quality  of  performance,  rather  than  the  traditional 
quantitative/financial  measures,  such  as  headcount  and  cost  to  spend  a  dollar.  CEOs 
were  mostly  interested  in  the  quality  of  the  supplies  and  the  quality  of  the  relationships 
purchasing  personnel  establish  with  both  suppliers  and  internal  customers.  CPOs,  on  the 
other  hand,  is  interested  in  the  day-to-day  operations  of  their  office.  It  seems  odd  that 
CPOs  feel  that  they  are  measured  on  the  savings  they  accrue  from  price  negotiations, 
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while  the  CEO  really  only  cares  about  the  quality  of  the  product  or  service  delivered. 
This  contrast  can  many  times  lead  to  opposite  results.  By  driving  down  the  price  for  a 
product  or  service,  the  CPO  may  be  reducing  the  quality  received.  On  the  other  hand, 
ignoring  cost  of  operations  and  only  concentrating  on  supplier  relationships  could  lead  to 
significant  inefficiencies  and  higher  costs,  which  may  all  of  a  sudden  become  a  very 
important  measure  to  a  CEO.  The  balancing  act  required  by  a  CPO  is  a  very  challenging 
one.  He/she  must  focus  on  how  they  are  evaluated  by  their  bosses  and  then  develop 
operating  procedures  that  will  ensure  efficient  internal  operations  while  satisfying  the 
CEO  and  other  internal  customers.  It  should  also  be  pointed  out  that  CEOs  and  CPOs  for 
most  part  are  not  required  to  be  socially  economical  when  entering  into  contracts.  They 
obviously  do  not  have  the  same  goals  and  requirements  that  a  public  contracting  office 
has  in  meeting  small  business  and  other  socially  driven  programs. 

In  research  (Purchasing’s  Organizational  Roles  and  Responsibilities,  Fearon,  PhD 
and  Leenders,  DBA,  CAPS  1996))  that  attempted  to  define  roles  and  responsibilities  in 
purchasing  offices,  it  showed  that  purchasing’s  current  involvement  in  major  corporate 
activities  shows  outsourcing  as  the  only  activity  with  a  moderate  involvement. 
Particularly  low  were  Government  relations,  international  counter- trade/offset  planning, 
marketing  planning  and  corporate  merger/acquisition/alliances.  In  addition,  the  same 
research  study  showed  that  the  most  common  roles  and  responsibilities,  in  descending 
order  of  importance,  of  a  purchasing  office  were: 

Establishing  policies  and  procedures. 

Executing  contracts  for  common  requirements. 

Participating  in  system- wide  purchasing/supply  personnel  decisions  and 

actions. 

Developing  supply  systems. 

C.  WHAT  METRICS  ARE  CURRENTLY  BEING  USED  TO  EVALUATE 
CONTRACTING  OFFICE  PERFORMANCE  AND  WHAT  ARE  THE 
LIMITATIONS  OF  THOSE  METRICS? 

The  following  is  a  list  of  the  current  Contracting  metrics  that  NAVSUP  is 
gathering  and  presenting  to  the  Chief  of  the  Navy  Supply  Corps  on  a  monthly  basis  as 
part  of  the  Chiefs  overall  “dashboard”  (see  the  Balanced  Scorecard  presented  in  Chapter 
II): 
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1 .  Number  of  Reverse  Auctions  conducted  and  saving  achieved. 

2.  Percentage  of  Performance  Based  Service  Contracts  (PBSC). 

3.  Percentage  of  work  Competed. 

4.  Contract  Closeout  and  the  number  of  over-aged  contracts. 

5.  Navy  and  Marine  Corps  (NMCI)  -  Meeting  Contract  Obligation 

Requirements. 

6.  Small  Business  -  Perfonnance  Evaluation  Summary. 

7.  Customer  Satisfaction. 

8.  Cycle  Time. 

9.  Procurement  Management  Review  (PMR)  reviews  as  they  relate  to 

Purchase  Card. 

Per  discussions  from  NAVSUP,  the  source  of  most  of  their  metrics  is  the  Field 
Contracting  Offices  and  the  Naval  Inventory  Control  Points  (Reverse  Auctions,  Contract 
Closeout,  NMCI,  Cycle  Time,  PMR  reviews).  Contract  closeout  is  a  mixture  of  MOCAS 
data  and  field  input.  Some  of  the  information  is  gathered  via  PMRS  (Competition,  Small 
Business,  and  PBSC),  and  Customer  Satisfaction  is  done  via  a  NAVSUP  web  site  that 
customers  log  onto  and  complete. 

Appendix  A  represents  the  current  metrics  FISC  Norfolk  utilizes  in  measuring 
contracting  perfonnance.  FISC  Norfolk  has  developed  a  balanced  scorecard  approach  in 
determining  the  efficiency  and  effectiveness  of  the  contracting  operation.  The  scorecard 
has  three  key  success  factors  (Effectiveness,  Efficiency  and  Sustainment): 

1.  Effectiveness  Index:  This  consists  of  three  weighted  individual 

metrics: 

a.  Number  of  Customers  Profiled. 

b.  Catalogues  and  IDTC’s  on  Portal/EMALL. 

c.  Customer  Satisfaction  Index. 

2.  Efficiency  Index:  This  consists  of  four  weighted  individual 

metrics: 

a.  Number  of  Intranet  Hits. 

b.  Number  of  Customers  on  the  Portal/EMALL. 

c.  Electronic  Transactions  Index. 

d.  Number  of  Paperless  Review  Boards. 
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3 .  Sustainment  Index:  This  consists  of  four  weighted  individual 

metrics: 

a.  Series  standards  profded. 

b.  Percentages  of  IDPs  Executed. 

c.  Training  requirements  resourced. 

d .  Training  hours  met. 

The  scorecard  then  has  a  list  of  initiative  metrics.  Each  initiative  will  have  several 
initiatives  associated  with  them.  The  scorecard  will  capture  at  least  one  metric  for  them. 
The  standards  for  growth  metrics  are  typically  determined  by  the  fiscal  month  of  the  year, 

e.g.  in  December  we  should  have  accomplished  25%  of  the  goal;  or  by  the  initiatives  plan 
for  implementation.  Minimum  grades  or  Statistical  Process  Control  sets  the  standards  for 
process  metrics  as  appropriate.  “Red/Yellow/Green”  indicators  are  assigned  to  individual 
metrics.  The  initiative  metrics  are: 

a.  Customer  Satisfaction  Survey. 

b.  Portal  Customer  Market  Share. 

c.  Percentage  of  Performance  based  Contracts. 

d.  Reverse  Auctions. 

e.  Self-Assessment  Findings. 

f.  Small  Business. 

g.  Overage  Contract  Closeout. 

h.  SPS  Upgrade  Effectiveness. 

i.  Percentage  of  Customer  Requirements  met  via  the  Portal/EMALL. 

j.  Percentage  of  FISC  Market  Sourced. 

k.  NMCI  Order  Fulfillment  rates. 

l.  Employee  Satisfaction. 

m.  Alignment. 

1.  NAVSUP’s  Metrics 

The  first  thing  to  recognize  when  analyzing  both  NAVSUP  and  FISC  Norfolk’s 
metrics  is  what  was  said  back  in  Chapter  II,  namely,  “it  is  easier  to  develop  lower  level 
scorecards  that  link  together  if  the  metrics  for  the  CEO  have  been  defined  first”.  You  can 
develop  scorecards  at  many  different  levels  of  the  organization.  The  key  is  that  they  link 
to  higher-level  metrics  within  the  command  or  to  a  higher  echelon  command.  NAVSUP, 
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as  a  Systems  Command,  must  focus  on  strategic  and  in  most  cases  ASN  driven  metrics. 
FISC  Norfolk,  a  lower  echelon  is  interested  in  a  different  set  of  metrics  (combination  of 
strategic  and  operating  metrics).  That  being  said,  all  lower  level  metrics  or  operating 
metrics  should  feed  into  the  higher  level.  Unfortunately,  NAVSUP’s  metrics  appear  to 
be  merely  counting  and  “putting  out  the  fire”  type  metrics.  For  instance,  how  many 
Performance  Based  Service  Contracts  have  you  awarded?  How  many  overage  closeouts 
have  you  performed?  What  is  your  Competition  rate?  What  are  your  Small  Business 
Statistics?  How  many  reverse  auctions  have  you  conducted  and  what  are  the  savings? 
How  many  PMRs  were  conducted  and  how  many  received  an  unsatisfactory  rating?  One 
can  argue  that  none  of  these  metrics  measure  the  efficiency  of  a  contracting  office.  Some 
may  allude  to  the  effectiveness  of  the  contracting  office  (customer  satisfaction,  NMCI 
Obligation  rates),  but  overall  it  would  be  difficult  to  determine,  just  based  on  these 
metrics,  if  your  contracting  function  is  efficient  or  effective.  For  example,  a  monthly 
counting  of  the  number  of  reverse  auctions  and  perfonnance  based  service  contracts, 
when  both  are  meant  to  be  contracting  tools  one  should  consider  when  determining  the 
best  approach  to  meeting  a  customer’s  requirements,  does  not  provide  you  with  an  insight 
as  to  whether  these  tools  were  used  appropriately  and  achieved  the  goal  of  satisfying  the 
customers.  There  are  two  mistakes  here. 

First,  establishing  reverse  auctions  as  a  dashboard  metric  for  the  Chief  of  the 
Naval  Supply  Systems  Command  does  not  seem  appropriate.  This  measure  does  not 
provide  an  accurate  barometer  as  to  whether  the  contracting  function  is  operating 
“efficiently  and  effectively  to  deliver  combat  capability  through  logistics”.  If  used  at  all, 
reverse  auctions  should  be  a  lower  level  metric. 

Second,  using  a  reverse  auction  goal  and  measuring  an  office  against  that  goal 

sends  the  wrong  message.  It  may  motivate  contracting  offices  to  fit  a  round  hole  into  a 

square  peg.  Reverse  auctions  may  be  an  excellent  tool  under  the  right  circumstances. 

The  key  is  to  ensure  that  your  contracting  officers  are  trained  and  know  the  value  of  all 

the  tools  available  to  them.  Performance  Based  Service  Contracts  (PBSCs),  Small 

Business,  and  Contract  Closeout  fall  into  this  same  category,  with  a  major  difference, 

ASN  has  mandated  these  metrics.  ASN  requires  that  50%  of  all  eligible  service  contracts 

must  be  performance  based  by  FY  05.  Again,  similar  to  reverse  auction,  one  can  argue 
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that  a  performance  based  service  contract  is  just  another  tool  for  a  contracting  officer  to 
consider  when  determining  the  best  way  to  satisfy  the  customer.  In  addition,  assigning  a 
performance  based  services  contract  goal  against  a  contracting  office  violates  one  of  the 
tenets  of  selecting  a  metric;  “the  metric  owner  has  the  ability  to  influence  the  metric”. 
While  contracting  offices  have  a  duty  to  train  requirements  generators  in  the  benefits  of 
performance-based  contracts,  they  do  not  define  the  actual  requirements.  Many  times 
the  PCOs  find  themselves  struggling  to  convert  a  non-performance  based  statement  of 
work  into  a  perfonnance  based  statement  of  work  while  still  satisfying  the  basic 
requirements  of  the  service  requested.  There  may  be  a  strong  argument  to  utilize  a 
performance  based  contract,  however,  as  GAO  pointed  out  (GAO  “Contract 
Management”:  Guidance  Needed  for  Using  Performance  Based  Service  Contracting), 
many  still  do  not  know  all  the  steps  required  in  defining  a  performance  based  service 
contract  and  more  guidance  is  needed  before  detennining  the  actual  benefit  of  this  type  of 
contract. 

NAVSUP’s  one  metric  that  lends  itself  to  efficiency  is  the  Cycle  Time  Metric. 
However,  this  metric  also  has  some  problems.  Cycle  time  measures  the  percentage  of 
time  you  award  a  contract/order  in  less  than  20  days  for  acquisitions  less  than  or  equal  to 
$25,000  and  30  days  for  acquisitions  between  $25,000-$  100,000.  One  of  the  problems 
associated  with  this  metric  is  performance  motivation.  On  many  occasions,  the  customer 
does  not  require  a  contract/order  within  the  20  or  30-day  goal  (finding  issues,  required 
delivery  dates,  sending  in  a  requirement  60  days  before  a  contract  is  needed).  On  the 
other  hand,  some  customers  need  their  requirements  filled  immediately.  PCOs  must  be 
capable  of  prioritizing  workload  without  having  to  worry  about  meeting  some  arbitrary 
goal  that  does  not  provide  a  measure  for  efficiency  and  can  seriously  hinder  your 
effectiveness.  Most  customers  would  not  consider  you  too  effective  if  you  awarded  their 
requirement  in  19  days  when  they  needed  it  in  2  days.  This  kind  of  metric  may  have  a 
negative  impact  on  customer  relations  and  motivate  poor  business  decisions.  Not  many 
managers  would  be  happy  to  know  that  their  PCOs  would  be  willing  to  give  up  on  the 
idea  of  negotiating  for  a  better  price  because  they  needed  to  get  an  award  done  within  the 
cycle  time  goals.  In  addition,  NAVSUP  did  not  consider  the  transaction  costs  associated 
with  gathering  and  managing  this  metric.  The  Standardized  Procurement  System  (SPS) 
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had  to  be  modified  and  negotiators  were  required  to  track  the  reasons  as  to  why  they  did 
not  meet  the  cycle  time  goals  and  these  had  to  be  reported  monthly.  The  only  way  to 
track  the  reasons  is  manually  and  is  labor  intensive.  One  has  to  question  the  value  in 
gathering  this  metric  when  you  analyze  the  cost  of  gathering  it. 

All  of  NAVSUP’s  metrics  were  established  with  no  analysis  of  the  management 
information  systems  available  at  each  of  their  field  offices.  SPS  does  not  have  the 
capability  to  provide  all  of  the  metrics  required.  Also,  all  of  NAVSUP’s  metrics  are 
“after  the  fact”  or  past  metrics  and  does  not  look  at  Work-in-Process  (WIP)  or  any  future 
metrics.  They  utilize  PMRS  and  MOCAS  for  some  metrics  and  manual  counts  for  others. 
One  of  the  basic  tenets  of  a  good  scorecard  is  to  have  a  metric  that  looks  as  the  past 
present  and  future  in  order  to  understand  trends  and  motivate  future  performance. 

2.  FISC  Norfolk’s  Metrics 

While  it  seems  many  of  NAVSUP’s  metrics  fail  to  measure  a  contracting  office’s 
efficiency  and  effectiveness,  FISC  Norfolk’s  metrics,  while  lacking  in  several  areas, 
makes  a  better  attempt  at  measuring  effectiveness  and  efficiency.  FISC  Norfolk  has 
adopted  the  balanced  scorecard  approach  as  espoused  by  Mark  Graham  Brown.  In  their 
Metrics  Manual  (Appendix  A),  FISC  Norfolk  emphasizes  that  performance  metrics 
should  encourage  performance  improvement,  effectiveness,  efficiency,  and  appropriate 
levels  of  internal  controls.  Where  practical,  they  should  incorporate  "best  practices" 
related  to  the  performance  being  measured  and  cost/risk/benefit  analysis,  where 
appropriate.  The  key  elements  of  the  performance  metrics  package  should  address  the 
following: 

•  Attaining  Organizational  Alignment. 

•  Improvements  in  quality,  cost  reduction  and/or  avoidance. 

•  Meeting  customer  requirements. 

•  Meeting  NAVSUP  requirements. 

•  Identification  and  development  of  resource  requirements. 

The  establishment  of  perfonnance  goals  can  best  be  specified  when  they  are 
defined  within  three  primary  levels: 
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•  Strategic  Objectives:  Broad,  general  areas  of  review.  These  generally 
reflect  the  highest  goals  based  upon  mission  accomplishment  and  vision 
achievement.  These  metrics  will  comprise  the  corporate  level  metrics. 

•  Operational  Criteria:  Specific  areas  of  accomplishment  that  satisfy  major 
divisions  of  responsibility  within  a  function.  These  metrics  are  generally 
mission  focused  and  form  the  underpinning  of  strategic  metrics. 

•  Warning  Measures:  Metrics  designed  to  drive  improvement  and 
characterize  progress  or  problems  within  each  operation.  These  are 
specific  quantifiable  goals  based  on  individual  expected  work  outputs  of 
critical  processes  or  contributory  functions. 

The  SMART  test  is  frequently  used  to  provide  a  quick  reference  to  determine  the 
quality  of  a  particular  performance  metric: 

S  =  Specific:  clear  and  focused  to  avoid  misinterpretation.  Should  include 
measure  assumptions  and  definitions  and  be  easily  interpreted. 

M  =  Measurable:  can  be  quantified  and  compared  to  other  data.  It  should  allow 
for  meaningful  statistical  analysis.  Avoid  "yes/no"  measures  except  in  limited  cases,  such 
as  start-up  or  systems-in-place  situations. 

A  =  Attainable:  achievable,  reasonable,  and  credible  under  conditions  expected. 

R  =  Realistic:  fits  into  the  organization's  constraints  and  is  cost-effective. 

T=  Timely:  doable  within  the  time  frame  given. 

However,  like  NAVSUP,  in  many  respects,  FISC  Norfolk  has  failed  to  adequately 
measure  the  effectiveness  and  efficiency  of  their  operation.  The  detachments  that  make 
up  the  FISC  Norfolk  Acquisition  Group  (Philadelphia,  Washington,  Hampton  Roads  and 
Norfolk  Naval  Shipyard)  do  not  share  the  same  management  information  system.  They 
are  currently  attempting  to  get  all  of  their  detachments  under  the  Philadelphia  Oracle 
based  management  information  system.  Failure  to  have  a  standardized  system  can  create 
inequities  and  in  some  occasions  require  manual  counts  of  data.  Before  a  metric  is 
established,  one  should  ensure  that  all  participants  have  the  ability  to  simply  measure  it 
via  a  report  out  of  a  database.  Unfortunately,  some  of  the  metrics  established  by  FISC 
Norfolk  does  not  look  at  efficiencies  or  effectiveness  because  some  of  the  contracting 
offices  do  not  have  the  ability  to  easily  gather  these  metrics.  The  researcher  is  providing 
a  model  as  part  of  his  recommendations.  The  model  proposed  is  assuming  that  all  offices 
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have  the  same  management  information  system.  This  is  a  required  element  of  the  model 
because  it  will  ensure  that  all  have  the  capability  of  measuring. 

Some  of  the  problems  with  FISC  Norfolk’s  metrics  are  the  complexity  of  the 
measures.  Many  individuals  the  researcher  interviewed  understood  the  importance  of 
gathering  a  specific  measure,  but  did  not  understand  how  they  were  being  graded  against 
that  metric.  For  instance,  on  page  1 1  of  Appendix  A,  FISC  Norfolk  provides  a  sample  of 
the  efficiency  index.  It  was  difficult  to  understand  where  the  figures  for  Data  and  Score 
came  from.  Most  understood  the  fact  that  you  would  have  different  weighting  assigned 
to  different  measures  as  some  are  more  important  than  others,  but  the  overall  metric’s 
scoring  sheet  was  sometimes  difficult  to  understand.  Also,  some  metrics  fall  into  the 
same  category  as  NAVSUP’s  in  that  they  measured  processes  that  FISC  Norfolk  had 
little  to  no  influence  over.  Specifically,  the  Portal/EMALL  metric  assumes  that  we  have 
the  ability  to  add  contractors  and  catalogs  to  the  EMALL.  While  it  is  true  that  we  may 
recommend  and  encourage  contractors  to  have  their  catalogs  added  to  the  EMALL,  we  do 
not  control  the  total  process  and  many  times  it  takes  months  in  the  DOD  EMALL  process 
chain  to  actually  get  a  contractor/catalog  added  to  the  site.  With  regards  to  this  metric, 
FISC  Norfolk  should  concentrate  only  on  the  areas  that  they  can  influence  (#  of 
customers  profiled)  and  disregard  the  measure  of  catalogs  and  Indefinite  Delivery  Type 
Contracts  (IDTCs)  added  to  the  site. 

Another  problem  with  the  current  metrics  is  the  Customer  Satisfaction  measure. 
This  measure  is  controlled  by  NAVSUP  and  requires  the  customers  to  access  NAVSUP’s 
website  and  complete  a  customer  satisfaction  survey.  Many  customers  have  expressed 
dissatisfaction  with  this  process  mainly  because  they  are  being  asked  on  a  quarterly  basis 
to  fill  out  a  satisfaction  survey.  In  addition,  there  are  many  different  customers 
associated  with  some  Commands  and  it  is  often  difficult  to  identify  what  specific 
customer  within  that  Command  needs  to  fill  out  the  survey.  Also,  on  some  occasions,  the 
sample  size  used  to  evaluate  customer  satisfaction  was  so  small  that  the  sample  could  not 
be  relied  on  to  provide  an  accurate  measure  of  the  effectiveness  of  customer  support. 

There  are  some  positives  to  the  current  FISC  Norfolk  balanced  scorecard.  For 
example,  FISC  Norfolk  examines  all  aspects  of  the  contracting  function,  including 
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employee  and  strategic  measures.  While  the  operational  and  customer  satisfaction 
measures  account  for  some  aspects  of  efficiency  and  effectiveness,  employee  measures 
help  the  organization  understand  the  overall  health  of  the  organization.  By  utilizing  this 
balanced  approach  the  organization  can  better  understand  where  they  need  to  adjust  in 
order  to  better  align  themselves.  They  can  focus  on  those  specific  areas  that  need 
additional  management  oversight.  This  has  been  most  beneficial  in  the  employee 
satisfaction  measure.  FISC  Norfolk  has  been  able  to  identify  specific  concerns  of  the 
employees  through  employee  surveys,  which  has  allowed  them  to  focus  their  efforts  in 
this  area.  Also,  FISC  Norfolk  has  developed  a  Quality  Assurance  Plan  (QAP)  that  has 
enabled  them  to  measure  the  quality  of  their  operations.  By  doing  quarterly  self- 
assessments,  the  organization  has  been  able  to  promulgate  on  an  enterprise  basis,  lessons 
learned  and  more  importantly,  identify  training  needs  of  the  workforce.  By  adopting  this 
approach  of  self-assessment,  FISC  Norfolk  has  established  a  very  robust  internal  training 
program  and  standardized  many  of  their  internal  operations,  which  has  led  to  a  more 
efficient  contracting  function.  FISC  Norfolk  has  accomplished  this  by  establishing  an 
INTRANET,  also  known  as  the  FISC  I-Web,  for  the  acquisition  function.  The  FISC 
Norfolk  acquisition  group  intranet  is  a  secure,  internal  website  designed  for  use  by  all 
acquisition  personnel  for  convenient  access  to  contracting  and  administrative  information 
in  a  web  based  environment.  The  I-Web  is  an  information  resource  that  provides  all 
necessary  web  sites/links  and  infonnation  to  support  the  acquisition  function  in  a 
centralized  location.  The  I-Web  provides  quick  access  to  and  sharing  of 
documents/policy  updates  and  many  acquisition  issues  that  are  important  to  the 
contracting  community.  By  standardizing  operations,  all  contracting  personnel  now 
utilize  the  same  clearance  templates,  and  have  instant  access  to  lessons  learned.  The  I- 
Web  also  acts  as  a  centralized  area  for  communities  of  expertise  exchanges,  where 
negotiators  can  have  instant  access  to  expert  advice.  FISC  Norfolk  also  utilizes  the  I- 
Web  to  post  results  of  the  balanced  scorecard.  This  allows  employees  to  see  how  the 
contracting  function  is  perfonning  and  more  importantly,  it  acts  as  a  communication  link 
between  upper  management  and  the  working  level.  FISC  Norfolk  provides  monthly 
metric  briefing  to  their  workforce,  which  promotes  understanding  and  feedback.  In 
Chapter  IV,  the  researcher  will  propose  a  model  (Appendix  B)  that  addresses  the 
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weakness  of  the  current  metric  process  to  go  along  with  some  of  the  strengths  of  the 
current  process. 

D.  DEFENSE  CONTRACT  MANAGEMENT  AGENCY  (DCMA)  METRICS 

As  a  member  of  a  contracting  office  that  handles  requirements  from  their 
inception  (Pre-Award)  until  their  disposal  (Post  Award),  the  researcher  was  interested  in 
discovering  what  metrics  DCMA  used  in  evaluating  their  post-award  function.  The 
following  information  was  given  to  the  researcher  from  the  Philadelphia  regional  DCMA 
office.  DCMA  looks  at  nine  metrics.  The  metrics  are  listed  by  title,  definition,  intent, 
computation  and  data  source: 

1.  Title:  Prime  Contracts  On-hand 

Definition:  The  quantity  of  contracts  assigned  for  primary 

administration  at  the  end  of  the  report  period. 

Intent:  This  metric  is  used  to  track  workload  data  for  internal  and 
external  customers. 

Computation:  The  sum  of  all  prime  contracts  on-hand  as  the 
contract  administration  office  at  the  end  of  the  report  period. 

Data  Source:  Mechanization  of  Contract  Administration  Services 
(MOCAS). 

2.  Title:  Obligated  Amount  of  Prime  Contracts  On-Hand 

Definition:  The  obligated  dollar  amount  of  prime  contracts 

assigned  for  administration  at  the  end  of  the  period. 

Intent:  This  metric  provides  workload  data  for  internal  and  external 
customers. 

Computation:  The  sum  of  the  obligated  dollar  amounts  of  all 
prime  contracts  on-hand  at  the  end. 

Data  Source  MOCAS. 

3.  Title:  Unliquidated  Amount  of  Prime  Contracts  On-Hand 

Definition:  The  unliquidated  dollar  amount  of  open  prime 

contracts  assigned  for  administration  at  the  end  of  the  period. 

Intent:  This  metric  is  used  to  track  workload  data  for  internal  and 
external  customers. 

Computation:  The  sum  of  the  unliquidated  dollar  amount  of  all 
prime  contracts  on-hand  at  the  end  of  the  report  period. 

Data  Source:  MOCAS. 
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4.  Title:  Percent  of  Schedule  On-Time 

Definition:  The  percent  of  line  item  schedules  delivered  before  or 
during  the  month  of  their  original  delivery  dates.  The  term  “line 
item  schedule”  refers  to  a  Contract  Line  Item  (CLIN)/subCLIN 
with  a  single  deliverable  group,  or  in  the  case  of  a  CLIN/subCLIN 
having  multiple  deliveries,  each  of  the  individual  groupings 
created  to  account  for  differing  destinations  and/or  delivery  dates. 

Intent:  On-time  deliveries  enable  customers  to  maintain  readiness 
requirements. 

Computation:  Divide  the  quantity  of  line  item  schedules  due 
during  the  report  month  that  were  delivered  during  or  before  the 
report  month  by  the  total  quantity  of  line  item  schedules  due 
during  the  report  month.  Multiply  the  result  by  100.  Use  the 
original  delivery  date  of  the  contract  if  the  contract  was  not 
modified  due  to  a  Government  caused  delay  OR  the  most  recent 
modified  delivery  date  due  to  a  Government  caused  delay.  In 
MOCAS  this  is  the  “Delivery  Schedule  Date”  field. 

Data  Source:  MOCAS  and  an  Internal  Management  Information 
System. 

5.  Title:  Outstanding  Delinquent  Line  Item  Schedule  Reduction 

Definition:  The  number  of  undelivered  line  item  schedules  whose 
current  delivery  dates  have  past  and  whose  entire  required  quantity 
has  not  been  shipped. 

Intent:  An  unacceptable  large  number  of  outstanding  schedules 
reside  in  our  database  systems.  Many  of  these  are  data 
misrepresentations.  Others  are  legitimate  delinquencies. 
Erroneous  data  represents  true  contractor  performance  to  our 
buying  activity  customers  and  is  an  unreliable  input  to  contractor 
risk  assessments  and  subsequent  surveillance  activity.  Legitimate 
delinquencies  hinder  the  readiness  requirements  of  the  Armed 
Services  and  require  top-level  visibility. 

Computation:  Count  the  line  item  schedules  in  which  the 

scheduled  quantity  exceeds  the  ship  quantity,  and  the  month  of  its 
current  delivery  schedule  is  past. 

Data  Source:  MOCAS. 

6.  Title:  Undefinitized  Contract  Actions  (UCAs)  On-Time 
Definitized  UCAs  and  Overage  UCAs 

Definition:  UCAs  not  definitized  within  180  days  for  the  date 
issued  are  overage. 

Intent:  To  reduce  the  overall  number  of  overage  UCAs. 
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Computation:  The  overage  unclefinitizcd  contract  action  rate  is 
calculated  by  dividing  the  quantity  of  overage  UCAs  by  the 
quantity  of  active  UCAs  during  the  reporting  period. 

Data  Source:  Pricing  and  Negotiation  AMS  module. 

7.  Title:  Contracts  Closed  On-Time 

Definitions:  Achieve  closeout  of  contracts  within  the  FAR 

mandated  timeframes. 

Intent:  To  close  al  contracts  87%  of  the  time  within  mandated 
timeframes  and  to  measure  our  contract  closure  similar  to  how  our 
customers  measure  our  contract  closeout  success.  Also,  to  allow 
for  cleanup  of  MOCAS  Section  2  without  being  penalized  by  the 
old  measurement  of  overage  percentage. 

Computation:  The  percentage  of  time  all  contracts  close  within 
their  FAR  mandated  timeframes. 

Data  Source:  MOCAS  Part  A  and  B,  Section  8. 

8.  Title:  Contracts  Past  Final  Delivery  Date 

Definition:  Monitor  percentage  of  contracts  past  their  final 

delivery  date  (FDD). 

Intent:  Maintain  contracts  more  than  180  days  past  their  final 
delivery  date  at  5%  or  less  as  the  end  of  the  month  and  to 
accurately  identify  all  physically  completed  contracts  for  our 
customers  and  ensure  that  these  contracts  are  in  MOCAS  Section 
2. 

Computation:  Percentage  of  Contracts  in  MOCAS  part  A,  Section 
1  that  are  more  than  180  days  past  their  FDD. 

Data  Source:  MOCAS  Part  A,  Section  1 . 


9.  Title:  Quantity  of  Overage  Contracts  for  Closure 

Definition:  Reduce  the  quantity  of  overage  contracts. 

Intent:  Reduce  the  quantity  of  overage  contracts  by  at  least  50%. 
In  addition  the  intent  of  this  metric  is  to  determine  that  the  quantity 
of  overage  contracts  does  not  disproportionately  increase  in  the 
efforts  to  meet  the  current  perfonnance  target  and  also  allow  for 
cleanup  of  MOCAS  Sections  without  being  penalized  by  the  old 
measurement  of  overage  percentage. 

Computation:  The  quantity  of  overage  is  computed  by 

determining  the  quantity  of  contracts  in  the  population  that  are 
overage  for  closure. 

Data  Source:  MOCAS  Part  A,  Section  2  overages. 
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The  researcher  considers  all  of  these  metrics  to  be  counting  metrics  and  provide 
little  to  no  insight  in  organizational  effectiveness  and  efficiency.  It  seems  that  DCMA 
has  a  good  handle  on  how  much  work  they  have,  but  no  idea  if  they  are  performing 
efficiently.  The  nine  metrics  listed  above  look  more  like  a  wish  list  of  strategic  goals 
(e.g.  reduce  the  quantity  of  overage  contracts).  The  problem  with  these  types  of  metrics 
is  that  you  may  never  know  if  you  are  doing  an  efficient  job  or  not.  For  example,  you 
may  be  doing  an  outstanding  job  in  closing  out  contracts,  but  the  number  of  overage 
contracts  may  increase  (due  to  current  contracts  becoming  overage)  leading  you  to 
believe  that  you  are  not  being  efficient.  They  do  not  address  the  overall  process.  A 
better  measure  may  be  to  focus  on  a  priority  system  for  closing  out  contracts,  with  an 
initial  focus  on  closing  out  contracts  whose  funding  expires  that  fiscal  year,  with  least 
important  priority  being  the  simple  fixed  price  instruments  you  can  easily  close  out.  By 
focusing  on  the  numbers  of  closeouts  you  have  accomplished  in  a  specific  month,  you  are 
motivating  employees  to  focus  on  the  “low  hanging  fruit”  and  not  the  more  critical 
closeouts  where  you  may  actually  be  able  to  recoup  funds  and  not  have  to  use  current 
year  funding  to  pay  for  past  year  performance.  DCMA’s  metrics  seem  to  mirror  the  top 
ten  measurement  mistakes  that  Mark  Graham  Brown  points  out  in  his  book  “Winning 
Score”.  They  provide  no  real  insight  into  the  effectiveness  or  efficiency  of  the  contract 
administration  function. 

What  should  DCMA  consider  when  measuring  their  processes?  DCMA  should 
consider  the  same  things  that  FISC  Norfolk  and  NAVSUP  need  to  consider.  Mission 
related  metrics  should  meet  the  following  characteristics  (Graham  Brown,  “Winning 
Score”,  2000,  p.  48): 

1 .  There  are  no  more  than  20  metrics. 

2.  Metrics  are  important  and  link  back  to  overall  goals  and  mission 
statements. 

3.  Metrics  are  a  roughly  equal  mix  of  past,  present  and  future. 

4.  Counting  measures  are  based  upon  meaningful  things,  where  possible. 

5.  Judgment  metrics  are  based  upon  ratings  established  using  specific 
criteria. 
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6.  Metrics  are  as  objective  as  possible. 

7.  Metrics  will  drive  the  right  behaviors  from  employees  and  partners. 

8.  Collecting  data  on  the  metric  is  feasible  and  cost  effective. 

9.  Employees  can’t  easily  cheat  on  measures. 

10.  Most  employees  can  understand  measures. 

1 1 .  Measures  lend  themselves  to  be  tracked  on  a  regular  basis  so  that  changes 
can  be  detected  in  perfonnance  when  there  is  still  time  to  do  something 
about  it. 

12.  Measures  depict  aspects  of  perfonnance  over  which  we  have  quite  a  bit  of 
influence. 

These  recommended  tips  should  be  examined  when  metrics  are  established.  It  provides 
the  metrics  manager  a  checklist  to  ensure  the  proposed  metrics  are  adequate  and  meet  the 
needs  of  the  organization  as  well  as  help  determine  which  metrics  need  to  be  added  or 
deleted. 

E.  CHAPTER  SUMMARY 

This  chapter  has  provided  the  results  and  analysis  of  the  data.  The  results  from 
the  research  effort  revealed  affirmation  of  the  literature  review.  Agencies  are  still 
struggling  to  establish  meaningful  metrics  and  are  having  difficulty  in  meeting  the 
requirements  of  the  Government  Performance  Results  Act.  The  research  also  indicated 
that  the  Navy  has  not  ever  measured  the  efficiency  of  a  contracting  office  (to  the  best  of 
the  researcher’s  knowledge)  and  believes  based  on  the  results  of  their  studies,  that  all 
contracting  personnel  are  alike  and  have  the  same  skill  sets  required  to  satisfy  all  types  of 
requirements. 

Next,  the  chapter  examined  the  results  of  a  study  performed  in  the  private  sector 
that  focused  on  CEOs  and  CPOs.  The  results  showed  some  significant  dichotomies 
between  what  a  Chief  Purchasing  Office  considers  important  to  the  effectiveness  of  a 
purchasing  office,  compared  to  what  a  Chief  Executive  Officer  finds  relevant.  While 
CEOs  were  more  interested  in  the  quality  of  the  supplies  and  the  quality  of  the 
relationships  purchasing  personnel  establish  with  both  suppliers  and  internal  customers. 
CPOs,  were  interested  in  the  savings  they  accrue  from  price  negotiations.  This  contrast 
can  many  times  lead  to  opposite  results.  By  driving  down  the  price  for  a  product  or 
service,  the  CPO  may  be  reducing  the  quality  received. 
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The  chapter  then  focused  on  the  current  NAVSUP  and  FISC  Norfolk  metrics. 
The  research  pointed  out  many  limitations  in  the  current  metrics.  The  research  also 
showed  some  positive  aspects  of  FISC  Norfolk’s  current  process. 

The  DCMA  metrics  were  then  examined  to  gain  a  better  understanding  of  the 
post-award  measures  that  the  administrative  contacting  community  employs.  The 
research  found  some  problems  with  DCMA’s  current  metrics.  Many  of  DCMA’s 
measures  do  not  address  efficiency  and  effectiveness.  Instead,  they  only  “count”  how 
many  things  they  are  doing. 

Finally,  the  results  obtained  during  this  research  effort  highlight  the  fact  that 
measuring  efficiency  and  effectiveness  of  a  contracting  office  is  an  extremely  difficult 
endeavor.  Mixing  subjective  measures  with  objective  measures  is  difficult  and 
sometimes  unfeasible.  Balancing  the  needs  and  desires  of  stakeholders  make  it  extremely 
difficult  to  achieve  organizational  efficiency  and  effectiveness.  A  comprehensive 
instrument  or  model  needs  to  be  developed  to  help  contracting  offices  measure  how  well 
they  are  performing. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

This  research  effort  provides  an  examination  into  the  contracting  function  with  a 
specific  emphasis  on  efficiency  and  effectiveness.  The  research  effort  used  a  qualitative 
approach  to  gathering  and  analyzing  data  from  a  Navy  and  private  industry  perspective. 
While  the  analysis  of  the  Navy  Studies  were  based  on  summaries  provided  by  a  private 
contractor,  it  still  provided  enough  information  to  analyze  what  and  why  the  Navy  has 
invested  significant  resources  in  studying  the  contracting  function.  The  research  also 
showed  the  limitations  of  these  studies.  Based  on  the  research,  the  following  conclusions 
are  supported  by  the  data  gathered: 

1.  The  Navy  Has  Not  Performed  An  Efficiency  or  Effectiveness 
Evaluation  of  a  Field  Contracting  Office 

The  research  indicates  that  the  Navy  studies  did  not  limit  themselves  to  a  field 
contracting  office,  in  fact,  most  of  the  studies  were  performed  to  determine  the  most 
effective  way  to  organize  the  contracting  function  in  the  Systems  Commands  or  the  Navy 
overall.  The  research  also  indicates  that  most  of  the  studies  draw  on  anecdotal  evidence 
and  therefore,  they  only  generally  address  and  hypothesize  on  the  efficiency  and 
effectiveness  of  a  contracting  office.  The  studies  provide  a  background  into  what  the 
Navy  leadership  considers  relevant  when  assessing  the  effectiveness  of  its  contracting 
function  but  they  did  not  examine  a  contracting  office  as  an  independent  entity,  nor  did 
they  attempt  to  measure  the  productivity  of  a  contracting  office.  Finally,  the  research 
indicates  there  still  seems  to  remain  a  perception  that  all  contracting  professional  (also 
known  as  “1102s”)  are  created  equally  with  the  same  skill  mix  to  acquire  any  type  of 
supply  or  service. 

2.  Purchasing  Managers  Do  Not  Always  Understand  How  They  Are 
Being  Evaluated 

An  analysis  of  the  private  sector  revealed  that  CEOs  and  CPOs  have  different 
opinions  when  it  comes  to  determining  the  efficiency  and  effectiveness  of  a  purchasing 
office.  This  research  showed  the  private  industry  struggles  as  much  as  the  public  sector 
in  detennining  what  measures  should  be  used  in  detennining  the  efficiency  and 
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effectiveness  of  a  purchasing  function.  The  research  showed  that  CEOs  are  mainly 
interested  in  measuring  the  effectiveness  of  a  purchasing  office,  with  less  emphasis  on 
efficiency.  They  tend  to  leave  that  up  to  the  Purchasing  Manager.  Specifically,  CEOs 
were  interested  in  the  quality  of  the  supplier’s  product  as  well  as  the  relationship  fostered 
with  the  supplier.  In  the  meantime,  the  research  indicated  that  CPOs  were  interested  in 
saving  money  and  getting  better  deals  for  the  organization  and  did  not  focus  on  the 
quality  of  the  supplier. 

On  the  public  side,  the  Navy  and  DCMA  share  the  same  problems  that  the  private 
sector  face.  The  measures  that  are  currently  employed  do  little  in  detennining  the 
effectiveness  and  efficiency  of  a  contracting  office.  Many  of  the  current  metrics  motivate 
inefficient  perfonnance  and  leaves  mangers  of  field  contracting  offices  struggling  to 
understand  what  they  should  be  doing. 

3.  Efficiency  and  Effectiveness  are  Difficult  to  Measure  at  the  Systems 
Command  and  the  Field  Contracting  Office  Level 

The  research  demonstrated  that  Systems  Commands  and  Field  Offices  have 
different  perspectives  when  it  comes  to  measuring  efficiency  and  effectiveness.  The 
research  indicates  that  the  current  metrics  required  by  NAVSUP  does  not  measure  the 
effectiveness  and  efficiency  of  a  contracting  office  and  fails  in  many  respects  to  meet  the 
requirements  of  the  GPRA.  The  research  also  showed  that  many  of  the  ’’counting” 
metrics  currently  deployed  come  from  higher  levels  within  the  Government  (ASN  and 
SBA)  and  they  have  different  motives,  mostly  political,  for  their  actions.  The  research 
inferred  that  many  times  qualitative  metrics  like  efficiency  and  effectiveness  should  only 
be  captured  at  the  operational  level  and  the  higher-level  metrics  should  focus  only  on 
mission  and  strategy.  The  research  indicates  that  the  Navy  has  a  difficult  time  linking 
their  metrics  to  their  overall  strategy. 

4.  The  Navy  Does  Not  Have  a  Model  to  Measure  the  Effectiveness  and 
Efficiency  of  a  Contracting  Office 

Based  on  the  results  of  the  research  conducted  concerning  the  definitions  of 
efficiency  and  effectiveness,  the  Navy’s  current  measurement  process  does  not 
adequately  measure  its  contracting  offices.  To  the  best  of  the  researchers  knowledge, 
there  is  no  indication  that  the  Navy  has  a  model  that  evaluates  all  aspects  of  the 


66 


contracting  office.  The  research  shows  many  holes  in  the  current  metrics  throughout  the 
hierarchy.  Additionally,  there  is  no  enterprise-wide  metrics  employed  by  the  Navy  to 
measure  contracting  offices  against.  This  void  creates  considerable  problems  when 
measuring  the  contracting  function.  The  research  of  the  many  studies  conducted  by  the 
Navy  indicates  that  the  hierarchy  may  not  be  interested  in  how  efficient  a  contracting 
office  performs.  Instead  it  appears  that  they  are  more  interested  in  appeasing  the  interests 
of  their  many  stakeholders. 

B.  RECOMMENDATIONS 

Based  on  the  results  and  analysis  of  the  data  reviewed,  several  recommendations 
for  selecting  the  proper  metrics  were  presented  in  Chapter  III.  These  recommendations 
should  be  followed  when  detennining  what  metrics  a  contracting  office  should  employ  to 
measure  its  effectiveness  and  efficiency.  The  researcher  is  proposing  a  model  that  is 
based  on  information  obtained  at  FISC  Norfolk  Detachment  Philadelphia.  The  model  is 
based  on  a  balanced  scorecard  approach  that  divides  the  organizational  metrics  into  4 
different  areas:  Strategic,  Operational,  Customer  and  Employee.  All  four  areas  are  equal 
in  importance  and  the  metrics  provided  in  each  area  must  be  continually  updated  to 
ensure  success.  The  Strategic  area  will  consist  of  several  kinds  of  metrics  that  address  the 
overall  output  of  the  organization  (obligation  and  action  rate)  as  well  as  strategic  mission 
metrics  (how  do  you  support  the  war  fighter)  and  vision  metrics  (actions  and  obligations 
per  Federal  Supply/Service  Code  (FSC)  and  other  strategic  type  metrics.  The  operational 
area  will  look  at  a  current  breakdown  of  work-in-process  by  type  of  action  and  will  depict 
a  manage-to-milestones  approach  that  will  be  explained  later.  The  customer  metrics  will 
focus  on  the  type  of  support  we  provide  for  the  customer  including  a  spend  analysis  of 
their  requirements  and  any  new  initiatives  we  are  providing  for  them.  Since  NAVSUP 
gathers  the  customer  satisfaction  metric  for  its  field  contracting  offices,  this  metric  will 
not  be  included  in  the  field  model.  The  employee  metrics  will  focus  on  available  training 
(courses  and  dollars),  intern  program  management  and  significant  employee  issues  and 
assignments. 

A  basic  requirement  for  using  this  model  is  that  you  must  have  a  management 

information  system  that  allows  you  to  extract  the  proper  metrics.  FISC  Norfolk 

Detachment  Philadelphia  uses  an  Oracle  based  management  information  system  called 
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ORACIMS.  ORACIMS  allows  one  to  measure  processes  to  significant  milestones, 
which  is  important  to  a  field  activity  like  FISC  Norfolk  Detachment  Philadelphia.  A 
brief  description  of  the  detachment  is  provided  so  one  can  better  understand  the  metrics 
model  that  is  proposed  as  Appendix  (C).  FISC  Norfolk  Detachment  Philadelphia  is  a 
Navy  field  contracting  office  that  provides  contracting  services  to  over  175  customers.  A 
significant  majority  of  the  actions  done  by  this  detachment  are  in  the  services  area  and 
are  also  high  dollar  type  requirements.  The  detachment  annually  obligates  approximately 
one  billion  dollars.  Approximately  2%  or  20  million  is  obligated  in  the  Simplified 
Acquisition  Threshold  ($0-$  100,000)  area.  Because  Philadelphia  has  such  a  significant 
amount  of  Large  Procurements  (over  $100,000),  their  internal  management  information 
system  is  based  on  a  management  to  milestones  philosophy  that  divides  all  new 
procurement  work  into  four  areas: 

1 .  The  Unsolicited  Phase. 

2.  Solicitation  Phase. 

3.  Technical  evaluation  Phase. 

4.  Negotiation  Phase. 

The  management  information  systems  also  track  all  requirements  form  “cradle  to 
grave”  and  has  the  capability  to  track  many  types  of  administrative  actions  as  well  as 
unmeasured  work  (the  system  was  recently  updated  so  buyers  could  keep  track  of  the 
amount  of  time  they  spend  on  acquisition  planning,  answering  administrative  issues, 
customer  meetings).  This  has  become  important  recently  because  of  the  Navy 
“Transformation”  effort  underway.  It  provides  NAVSUP  and  ASN  an  idea  of  the  amount 
of  time  dedicated  to  unmeasured  workload.  The  researcher  believes  that  the  model  being 
presented  in  Appendix  B  will  go  a  long  way  in  measuring  the  efficiency  and  effectiveness 
of  a  field  contracting  office. 

The  following  provides  an  explanation  of  the  metrics  displayed  in  Appendix  B: 

1.  Strategic  Metrics 

Actions  and  Obligations:  The  first  2  slides  compare  actions  and 
obligations  over  the  past  2  fiscal  years.  This  gives  managers  a  picture  of 
the  current  year  obligation  rates  and  gives  them  trend  data.  The  data 
source  for  this  metric  is  PMRS. 


68 


Large  Purchase  Top  Ten  Suppliers  by  Dollars:  Managers  can  use  this 
type  of  information  for  leveraging  with  suppliers  on  future  requirements. 
The  data  source  for  this  metric  is  PMRS. 

Actions  and  Obligations  for  Services/Supplies:  Provides  managers  an 
indication  of  the  type  of  work  you  are  perfonning.  There  is  a  significant 
difference  between  service  contracting  and  supply  contracting.  Much 
more  oversight  is  required  in  a  services  contract  and  for  the  most  part  our 
office  retains  contract  administration  over  the  service  contracts  we  issue. 
It  is  important  that  managers  understand  the  significance  of  this  metric. 
The  data  source  for  this  metric  is  PMRS. 

Large  Purchase  Top  Ten  FSCs  by  Actions:  This  metric  gives  managers 
an  indication  of  the  types  of  services  they  are  buying.  A  corresponding 
metric  for  supplies  would  be  appropriate  if  your  office  procured  a  majority 
of  them.  While  additional  exploration  of  the  exact  type  services  is 
required,  it  still  provides  management  with  a  good  idea  of  the  type  of 
services  the  office  is  familiar  with  and  maybe  even  expertise  at  acquiring. 
This  type  of  metric  fits  into  the  strategic  enterprise  area  as  it  may  provide 
headquarters  with  insight  into  where  they  may  want  to  centralize  certain 
types  of  services  or  supplies  as  was  recommended  in  several  of  the  studies 
analyzed  in  Chapter  III.  The  data  source  for  this  metric  is  PMRS. 

Competition:  This  metric  is  included  in  the  Strategic  area  because  it  is 
required  by  NAVSUP.  A  month-to-month  compilation  is  recommended  to 
provide  trend  data.  While  NAVSUP  already  gathers  this  metric  for  their 
dashboard,  it  is  still  important  to  keep  it  at  the  field  level,  so  you  are  aware 
of  significant  changes  that  may  affect  your  goal.  The  data  source  for  this 
is  PMRS.  A  recommended  change  to  this  metric  is  to  calculate  the 
number  of  dollars  that  have  been  successfully  broken  out  of  the  sole 
source  area.  This  metric  can  be  gathered  via  the  ORACIMS  database  as 
long  as  the  negotiators  are  diligent  in  recording  this  type  of  transaction. 
The  researcher  would  also  recommend  some  recognition  to  those 
individuals  who  breakout  historically  sole  source  actions. 

E-Business:  This  metric  is  also  included  in  the  Strategic  area  because  it  is 
required  by  NAVSUP.  OSD  and  ASN  have  a  significant  interest  in 
making  contracting  paperless.  OSD  had  a  goal  to  make  all  contracting 
paperless  by  CY  2001.  Their  assumption  was  that  SPS  would  be  fully 
implemented  by  then  and  all  requirements  both  received  and  completed 
would  be  done  paperless.  Unfortunately,  DOD  has  never  agreed  on  an 
electronic  requirements  generating  system  and  not  all  awards  are  done  in 
SPS  for  various  reasons  (urgency,  NMCI  orders  are  exempt  from  SPS 
because  they  have  their  own  electronic  systems).  We  include  this  metric 
because  it  allows  us  to  focus  on  our  top  requirements  providers  and  jointly 
try  to  determine  a  way  to  receive  electronic  purchase  requests.  The 
problem  with  this  metric  is  the  data  sources.  Some  infonnation  is  pulled 
from  SPS,  while  the  rest  is  pulled  from  ORACIMS  (we  added  a  block  to 
our  ORACIMS  database  to  get  this  infonnation).  If  you  do  not  have  a 
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management  information  system,  you  will  have  to  manually  gather  this 
data. 

Anticipated  Future  Obligations:  Forecasting  future  obligations  and 
business  is  important  for  planning  and  resourcing,  especially  if  it  is  an 
indefinite  delivery  type  contract  where  we  anticipate  issuing  significant 
orders.  This  metric  is  intended  to  look  at  your  top  ten  future  obligations. 
The  data  source  for  this  metric  is  ORACIMS. 

Contract  Closeout:  This  metric  is  included  in  the  Strategic  area  because 
it  is  required  by  NAVSUP  and  ASN.  The  Under  Secretary  of  Defense  has 
a  significant  interest  in  reducing  the  amount  of  open  contracts.  The 
contract  closeout  status  metric  just  counts  the  amount  of  closeouts 
accomplished.  The  data  source  for  this  is  ORACIMS.  The  dollars 
deobligated  gives  management  an  idea  of  the  amount  of  money  returned  to 
the  customer.  This  also  helps  focus  the  employees  on  a  specific  area  in 
close-out  that  has  management  attention  and  hopefully  will  motivate  them 
to  closeout  those  contracts  where  there  is  a  possibility  for  recoupment 
before  they  focus  on  the  “low  hanging  fruit”.  The  data  source  for  this 
metric  is  ORACIMS  and  PMRS.  The  last  closeout  metric  is  required 
specifically  by  NAVSUP  and  must  be  updated  monthly.  The  data  source 
for  this  metric  is  MOCAS  and  ORACIMS. 

Small  Business  Goals:  Again,  this  metric  is  included  in  the  Strategic  area 
because  it  is  required  by  NAVSUP  and  ASN.  It  provides  management 
with  a  monthly  snapshot  of  how  they  are  faring  against  their  assigned 
goal.  The  data  source  for  this  metric  is  PMRS. 

Business  Development  and  Major  Program  Support:  While  these  two 
metrics  may  not  fit  into  a  measurement  of  efficiency  and  effectiveness, 
they  do  provide  management  with  a  monthly  picture  of  new  developments 
and  a  running  ledger  of  major  programs  supported.  The  metrics  are 
beneficial  when  they  are  communicated  to  the  employees  so  everybody  is 
familiar  with  the  type  of  programs  the  office  supports.  The  data  source  for 
this  metric  is  manual. 

In  summary,  the  purpose  of  the  strategic  metrics  is  to  give  management  insight  into  the 
various  strategic  interest  areas  of  NAVSUP  and  ASN,  while  also  focusing  on  the  field 
office. 

2.  Operational  Metrics 

Cycle  time  metrics:  This  metric  could  have  possibly  been  placed  in  the 
strategic  area  because  is  required  by  NAVSUP,  but  it  fits  better  into  the 
operational  area.  Again,  this  metric  has  the  possibility  of  promoting  poor 
business  decisions.  However,  used  properly,  it  can  give  you  insight  into 
possible  systemic  issues  if  you  are  consistently  missing  your  goal.  The 
key  to  this  metric  is  understanding  why  the  goal  is  missed  and  identifies 
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the  root  causes  for  missing  the  goal.  The  data  source  for  this  metric  is 
SPS. 

Competitive  Sourcing  and  Streamlined  A-76  Process.  This  metric 
could  also  fit  in  the  strategic  area  because  of  the  high  visibility  associated 
with  this  program.  The  researcher  recommends  it  be  placed  in  the 
operational  area  because  of  the  effect  the  A-76  process  has  on 
Procurement  Administrative  Lead  Time  (PALT),  which  is  an  efficiency 
type  metric.  Comparing  the  time  it  takes  to  d  a  full  study  compared  to  a 
streamlined  is  important  to  understand  and  communicate  to  higher  levels. 
It  is  also  an  important  metric  because  a  Government  win  results  in  no 
obligations  in  the  DD  350  system.  The  A-76  process  is  sometimes  a  long 
resource  intensive  process  that  receives  no  “credit”  in  the  PMRS  database. 
Therefore,  it  is  critical  to  have  a  total  understanding  of  the  workload  in 
this  area.  The  data  source  for  these  metrics  is  ORACIMS. 

Work-in-Process  Profile,  Acquisition  Strategy  and  Contract  Type: 

These  metrics  provide  management  with  an  instant  picture  of  what 
encompasses  their  current  workload.  By  breaking  the 
folders/requirements  out  by  new  procurement,  Type-I  Delivery  Orders 
(fixed  priced  orders  under  single  of  multiple  award  contracts),  Type  II 
delivery  orders  (Time  and  Material,  Labor  Hour,  Cost  Type  orders  under 
single  or  multiple  award  contracts)  and  other  provides  a  quick  picture  of 
the  type  of  work  in  process.  The  acquisition  strategy  metric  also  provides 
you  with  an  instant  picture  of  the  strategy  the  negotiator  will  be  employing 
in  meeting  the  requirements.  Tracking  by  contract  type  can  also  provide 
management  with  important  information  for  allocating  resources.  The 
large  amount  of  cost  type  contracts  may  require  moving  resources  around 
the  command  or  moving  work  depending  on  the  overall  strategy  of  the 
detachment.  As  stated  above,  usually  more  oversight  is  required  in  service 
contracting  and  especially,  cost  type  service  contracts.  The  data  source  for 
these  metrics  is  ORACIMS. 

New  Procurement  Summary:  This  is  where  the  management  to 
milestones  philosophy  comes  into  play.  In  an  organization  that  has  a 
significant  amount  of  large  dollar  complex  contracts,  it  is  important  to 
install  some  kind  of  milestone  system.  The  purpose  of  the  milestone 
system  is  to  ensure  that  we  are  attempting  to  operate  as  efficiently  as 
possible.  The  Philadelphia  Detachment  has  divided  the  miles  tones  into 
four  specific  phases:  The  Unsolicited  Phase,  Solicitation  Phase,  Technical 
Evaluation  Phase,  and  the  Negotiation  Phase.  In  addition,  the  new  folders 
are  further  delineated  by  Competitive,  Sole  Source,  Competitive 
Commercial  Items  and  Sole  Source  Competitive  Items.  The  reason  for 
this  separation  is  the  different  rules  that  apply  to  commercial  items  (FAR 
Part  13.  5  and  other  FAR  Part  12  rules).  This  is  just  an  overall  picture  of 
the  new  work-in-process.  The  next  four  metrics  provide  a  more  in-depth 
look  at  the  work-in-process.  The  data  source  for  this  metric  is  ORACIMS. 
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Unsolicited  Phase:  Management  has  established  a  baseline  that  all  new 
procurements  should  be  solicited  within  45  days  (30  days  for  commercial 
items)  after  receipt.  This  standard  represents  the  average  time  it  takes  to 
get  to  the  solicitation  phase  and  management  understands  that  with  good 
reason  this  standard  may  not  be  met.  Management  should  review  the 
standard  at  least  quarterly  and  adjust  as  appropriate.  The  detachment  uses 
a  red-yellow-green  rating  to  each  new  folder  in  this  milestone  area.  The 
purpose  is  to  encourage  the  workforce  to  solicit  their  requirements  as 
quickly  as  they  possibly  can  without  sacrificing  quality.  A  red  rating  may 
not  be  a  bad  thing  if  proper  justification  accompanies  it.  This  metric  may 
also  allow  managers  to  identify  some  systematic  issues  concerning  the 
unsolicited  phase  and  may  lead  to  fixing  root  cause  problems.  The  data 
source  for  this  metric  is  ORACIMS. 

Solicitation  Phase:  The  goal  of  the  detachment  is  to  have  solicitations 
close  in  45  days  or  less  (30  days  or  less  for  commercial  items).  Similar  to 
the  unsolicited  phase,  management  understands  that  there  may  be 
instances  that  this  can’t  be  met.  The  purpose  of  this  metric  is  to  gain 
insight  into  the  reasons  solicitations  are  delayed  and  resolve  any  root 
cause  problems.  The  data  source  for  this  metric  is  ORACIMS. 

Technical  Evaluation  Phase:  The  goal  of  the  detachment  is  to  have 
technical  evaluations  completed  in  45  days  or  less.  While  in  many 
instances  the  negotiator  can’t  control  the  time  it  takes  for  a  technical 
person  to  perform  his  duty,  it  is  important  for  management  to  understand 
what  causes  the  delay.  The  detachment  experienced  a  benefit  of  tracking 
this  metric.  They  found  the  root  cause  of  late  technical  evaluations  (on¬ 
site  vice  off-site  evaluations).  The  data  source  for  this  metric  is 
ORACIMS. 

Negotiation  Phase:  The  goal  of  the  detachment  is  to  conclude 
negotiations  within  45  days  after  the  solicitation  closes.  This  metric 
serves  in  the  same  capacity  as  the  other  three  mentioned  above.  The  data 
source  for  this  metric  is  ORACIMS. 

Delivery  Orders:  The  quantity  of  delivery  orders  being  written  today  are 
increasing  at  an  alanning  rate  and  has  become  a  significant  area  of  interest 
at  FISC  Norfolk  Detachment  Philadelphia.  What’s  also  growing  at  an 
alarming  rate  is  the  number  of  multiple  award  contracts.  This  metric  is 
broken  down  by  order  type  (I  or  II)  because  fixed  priced  delivery  orders 
(Type  I)  are  less  complex  than  cost  type  orders  (Type  II)  and  further 
divided  by  competitive  and  non-competitive.  This  is  an  important  feature 
of  the  delivery  order  process  as  some  competitive  delivery  orders  can 
require  technical  evaluation  factors  as  well  as  cost  and  prices  and  literally 
result  in  a  mini-competitive  best  value  source  selection.  The  standard  for 
non-competitive  Type  I  orders  is  15  days  or  less  and  45  days  or  less  for 
Type  II  orders.  The  goal  of  competitive  orders  is  45  days  or  less  no  matter 
what  type  of  order.  There  was  originally  one  standard  in  this  metric, 
however,  over  time  management  noticed  the  trend  associated  with 
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multiple  award  type  orders  and  decided  to  separate  them  out  so  they  have 
a  better  picture  of  the  delivery  order  process.  The  data  source  for  this 
metric  is  ORACIMS. 

Other  Procurement  Actions:  This  provides  management  with  the  rest  of 
the  workload  picture.  A  renewed  focus  on  contract  management  has  been 
undertaken  by  the  detachment  in  order  to  gain  a  better  understanding  of 
the  amount  of  time  and  resources  expended  in  contract  administration. 
Not  all  modifications  are  equal  and  the  goal  of  this  metric  is  to  separate 
out  the  simple  ones  (exercising  an  option  and  funding  action)  from  the 
more  complex  (Change  orders).  Simplified  Acquisition  Purchase  is  also 
captured  (most  of  these  are  simple).  This  provides  management  a 
snapshot  of  part  of  the  Contract  administration  picture.  The  source  data 
for  this  metric  is  ORACIMS. 

Large  Purchase  and  Simplified  Acquisition  Self-Assessment  Findings 
and  Self-Assessment  Recommendations:  It  is  great  to  have  a  picture  of 
the  entire  workload.  The  more  important  question  is  how  well  are  we 
doing?  What  is  the  quality  of  the  products  we  produce?  This  is  where  the 
quality  assurance  management  program  comes  in.  The  idea  is  to  conduct 
quarterly  self-assessments  on  at  least  10%  of  the  awarded  population  to 
gain  a  better  understanding  of  the  quality  of  the  contracting  operation. 
These  metrics  help  gauge  how  efficient  and  effective  an  organization  is 
performing.  The  first  step  in  the  process  is  to  establish  a  quality  assurance 
plan  that  identifies  the  key  acquisition  processes  in  your  command.  Then, 
after  you  define  the  key  acquisition  processes,  you  assign  a  standard  that 
you  expect  the  organization  to  achieve  in  that  process.  For  FISC  Norfolk 
a  team  determined  the  standards  as  acceptable  error  rates.  For  example,  it 
is  expected  that  90%  of  all  files  sampled  will  have  an  adequate  Business 
Clearance.  After  conducting  the  self-assessments  it  is  critical  to  post  the 
results  and  recommend  specific  actions  whether  it  be  a  change  in  business 
processes  or  additional  training.  This  metric  is  currently  gathered 
manually.  However,  an  automated  version  is  currently  being  studied. 

In  summary,  the  operational  metrics  are  focused  on  the  efficiency  of  the 
contacting  function.  The  model  suggests  a  measuring  to  milestones  approach  for  Work- 
in-process.  Most  of  the  operational  metrics  have  a  distinct  focus  on  the  present.  It  leaves 
the  past  metrics  to  the  strategic  and  customer  areas  and  concentrates  on  the  current 
operations. 

3.  Customer  Metrics. 

Large  Purchase  Top  Ten  Customers  by  Actions  and  Dollars:  These 
metrics  focus  on  the  top  ten  customers  either  by  action  or  by  dollars.  This 
gives  management  some  insight  into  what  customer  and  how  much  we  are 
supporting  them.  This  does  not  rank  customer  by  importance.  It  provides 
management  with  an  idea  of  the  resources  that  may  accompany  the  work 
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of  each  customer.  It  by  no  means  tells  the  entire  picture.  Some  customers 
have  small  quantities  of  actions,  but  may  be  critical  to  the  operations  of 
the  war  fighter.  The  purpose  of  this  metric  is  to  give  management  a  rough 
order  of  magnitude.  The  data  source  for  these  metrics  is  PMRS. 

What  do  we  buy  for  our  top  ten  customers:  These  metrics  focus 
specifically  on  what  we  acquire  for  our  customers.  These  metrics  should 
be  utilized  in  monthly  meetings  with  the  customer  to  determine  possible 
economies  by  combining  requirements  and  also  provide  the  customer  with 
a  little  portfolio  of  where  they  are  obligating  their  money.  These  metrics 
provide  a  beginning  framework  in  establishing  a  Customer  Relationship 
Management  (CRM)  Program.  In  addition,  these  metrics  could  be  used  on 
an  enterprise- wide  basis  not  just  for  NAVSUP  but  also  for  the  entire 
Navy.  The  data  source  for  these  metrics  in  PMRS. 

In  summary,  one  metric  that  seems  missing  in  the  customer  area  is  customer  satisfaction. 

Historically,  the  field  offices  use  to  survey  customers  quarterly  or  semi-annually.  In 

1999,  NAVSUP  decided  that  they  would  take  over  the  customer  satisfaction  survey  on  an 

enterprise-wide  basis.  NAVSUP  does  these  surveys  quarterly  and  provides  the  results  to 

the  individual  field  offices.  The  field  offices  have  found  some  flaws  to  this  process,  but 

do  not  wish  to  burden  customers  with  more  surveys  to  complete.  While  it  is  not  included 

in  any  of  the  proposed  metrics,  the  researcher  recommends  monthly  meetings  (any 

medium)  as  a  minimum  to  ensure  that  customer  expectations  are  being  met.  The 

researcher  was  not  able  to  develop  a  meaningful  metric  for  customer  meetings  other  than 

to  keep  track  of  them  and  report  out  any  significant  issues. 

4.  Employee  Metrics 

Staffing:  Although  this  is  a  “counting”  metric,  it  is  important  to 

understand  the  turnover  rate  of  your  organization.  The  researcher 
recommends  a  four-year  period.  Any  significant  spikes  in  the  chart  can 
alert  management  to  further  investigate  the  root  causes  of  departure.  This 
metric  may  provide  little  information  for  a  stable  workforce.  However, 
when  you  combine  that  with  the  training  budget  allocated  to  the 
workforce,  it  can  give  you  a  better  idea  of  how  much  training  dollars  are 
available  per  employee.  The  data  source  for  this  metric  is  the  human 
resource  personnel  database. 

Training  Statistics:  This  metric  provides  management  with  a  monthly 
snapshot  of  how  much  training  is  being  received  and  accounted  for  by  the 
workforce.  By  using  this  metric,  one  can  detennine  if  the  employees  are 
accounting  properly  for  the  amount  of  training  they  are  receiving.  The 
data  source  for  this  metric  is  manual  (sign-in  sheets). 
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Training  Budget  for  1102s,  1105s  and  non-1102s/1105s.  This  metric 
provides  management  with  a  picture  of  the  training  budget  and  how  well 
the  organization  is  implementing  its  external  training  program.  This 
metric  does  not  include  in-house  training  as  that  is  considered  a  no-cost 
training  program.  While  FISC  Norfolk  recommends  an  Individual 
Development  Plan  (IDP)  execution  rate,  the  researcher  does  not  support 
that  type  of  metric.  Based  on  prior  experience,  the  researcher  feels  that 
the  IDP  process  has  significant  flaws.  In  the  past  few  if  any  employees 
actually  received  the  training  they  requested  in  their  IDPs.  Many  of  the 
reasons  for  not  executing  IDPs  were  budget  driven  (training  dollars  are 
continually  dwindling)  and  most  employees  do  not  consider  the  IDP 
process  seriously.  They  are  tired  of  asking  for  specific  training  that  is 
endorsed  by  their  immediate  supervisor,  only  to  find  that  there  is  no 
funding  available  when  the  training  is  being  offered.  By  knowing  what 
the  training  budget  is  in  advance  of  the  fiscal  year,  managers  will  e  able  to 
allocate  training  dollars  per  employee.  This  will  help  employees  select 
training  courses  within  the  budget  allocated  to  them.  This  metric  will  help 
manage  the  training  budget  process.  The  data  source  for  this  metric  id 
One-touch  Financial  (OTF). 

Next  Generation  Workforce:  This  metric  keeps  track  of  all  of  the 
training  programs  and  employee  projects  and  possible  employee  issues.  It 
provides  management  with  an  idea  of  how  robust  the  training  program  as 
well  as  employee  involvement.  Included  in  this  metric  should  be  any 
employee  issues,  other  than  confidential.  The  data  source  for  this  metric  is 
manual. 

In  summary,  this  employee  area  addresses  the  past,  present  and  future.  Not  included  in 
these  metrics,  but  considered  important  are  employee  surveys.  Employee  surveys  can 
often  provide  insight  to  the  working  conditions  of  the  office.  However,  caution  should  be 
used  when  doing  employee  surveys.  One  of  the  major  flaws  associated  with  employee 
surveys,  as  experienced  by  the  researcher,  is  the  frequency  in  which  they  occur.  Monthly 
or  quarterly  surveys  are  not  recommended.  For  best  results,  the  researcher  recommends 
semi-annual  or  annual  employee  surveys.  The  most  important  aspect  of  any  survey  is 
that  you  perfonn  some  kind  of  action  in  response  to  the  survey.  To  do  nothing  is 
unacceptable  and  taints  the  importance  of  the  survey  as  a  measurement  tool. 

C.  ANSWERS  TO  RESEARCH  QUESTIONS 
1.  Primary  Research  Question 

a.  What  Model  Is  Needed  to  Measure  the  Efficiency  and 
Effectiveness  of  a  Navy  Field  Contracting  Office? 
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The  researcher  has  developed  a  model  (Appendix  B)  for  measuring  the 
efficiency  and  effectiveness  of  a  Navy  Filed  Contracting  Office.  The  model  is  based  on  a 
balanced  scorecard  approach  that  divides  the  organizational  metrics  into  4  different  areas: 
Strategic,  Operational,  Customer  and  Employee.  All  four  areas  are  equal  in  importance  and 
the  metrics  provided  in  each  area  must  be  continually  updated  to  ensure  success. 
Management  should  be  analyzing  the  metrics  on  a  monthly  basis  and  correct  any 
misalignment  of  the  organization.  This  will  also  provide  management  with  insights  into 
where  they  need  to  concentrate  their  resources.  For  example,  results  of  an  employee  survey 
that  show  employees  do  not  understand  where  they  fit  in  the  organizational  strategy  or  felt 
that  they  were  lacking  the  adequate  training  to  meet  the  mission  goals,  would  allow  managers 
to  concentrate  on  communicating  with  employees  and  map  out  a  training  program  that  would 
satisfy  their  needs.  The  model  also  measures  the  areas  that  are  required  form  NAVSUP  and 
ASN,  while  also  providing  the  field  office  with  there  own  set  of  measures. 

2.  Secondary  Research  Questions 

a.  What  is  the  Difference  Between  Efficient  and  Effective  and  Why 
Should  We  Strive  for  Both? 

As  was  discussed  in  Chapter  II,  efficiency  refers  to  the  capacity  to  produce 
results  with  the  minimum  expenditure  of  time,  money,  or  materials.  Efficiency  thus 
focuses  on  the  input-output  ratio.  To  be  efficient  is  to  do  things  well,  to  attend  to  the 
internal  organization  by  refining,  formalizing,  and  elaborating  on  existing  knowledge  and 
making  short-tenn  improvements.  Effectiveness  on  the  other  hand,  is  defined  as 
“productive  of  results”.  The  focus  is  on  doing  the  right  thing  and  that  is  determined  by 
outcome  attainment.  It  comes  from  an  understanding  and  interpretation  of  the  exterior 
environment  and  what  outcomes  are  required  (e.g.,  Customer  satisfaction,  goal 
achievement,  negotiated  savings). 

Both  efficiency  and  effectiveness  play  an  important  role  in  organizational 
performance,  yet  in  the  competition  for  resources,  each  can  interfere  with  the  other. 
Efficiency  depends  on  focus,  precision,  repetition,  analysis,  discipline  and  control. 
Effectiveness,  on  the  other  hand,  relies  on  experimentation,  novelty  and  loosened  control. 
While  effectiveness  thrives  on  experimentation,  efficiency  attempts  to  drive  out  any 
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uncertainties.  Consequently,  public  managers  need  to  make  choices  between  the  level  of 
efficiency  and  effectiveness  they  intend  to  pursue. 

Efficiency  looks  at  internal  operations,  while  effectiveness  is  viewed  as 
how  well  are  we  perfonning  by  the  external  environment  (stakeholders).  An  organization 
can  be  considered  quite  effective  by  a  customer,  yet  be  significantly  inefficient  when  you 
examine  their  internal  operating  procedures. 

So  why  do  we  need  to  strive  for  both?  Simply  stated,  we  are  required  to. 
As  stated  previously  in  this  chapter  “NAVSUP  has  embarked  on  a  journey  to  better 
structure  and  align  the  organization  to  most  effectively  and  efficiently  deliver  combat 
capability  through  logistics”.  In  addition  to  Navy  transfonnation,  the  goal  of  the  GPRA 
is  to  measure  us  by  our  outcomes  and  demonstrate  to  the  taxpayer  that  we  are  an  efficient 
and  effective  organization.  It  is  also  inherent  in  managing  any  type  of  business.  If  you 
want  to  keep  a  vibrant  and  successful  organization,  you  must  be  efficient  and  effective. 

b.  Where  Do  Most  Field  Contracting  Offices  Fit  in  the 
Organizational  Configuration  Model? 


HIGH 


EFFECTIVENESS 


LOW 


LOW 


HIGH 

EFFICIENCY 


Figure  4.  Field  Contracting  Officer  (FCO)  Organizational  Configuration 
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The  researcher  took  an  informal  survey  as  well  as  examined  several  field  contracting 

offices  mission  statement  and  strategic  plans  to  determine  where  they  fit  on  the 

organizational  configuration.  As  depicted  in  Figure  4,  most  see  themselves  as  a  hybrid 

organization.  Continuously  striving  for  customer  satisfaction  (effectiveness)  and  supporting 

the  war-fighter,  while  offsetting  that  with  ways  to  reduce  infrastructure  and  cycle  time 

(efficiency).  Most  Field  Contracting  Offices  find  themselves  being  graded  on  how  effective 

they  are  in  meeting  both  customer  needs  and  the  multiple  stakeholder  interests.  When  asked 

how  they  balance  efficiency  with  effectiveness  most  mentioned  the  balanced  scorecard  as  an 

approach  to  measuring  not  just  effectiveness  and  efficiency,  but  also  the  overall  performance 

of  the  entire  organization  in  how  it  meets  its  mission  and  supports  its  employees. 

c.  What  Metrics  Are  Typically  Used  to  Measure  Field  Contracting 
Office  Efficiency  and  Effectiveness  and  What  Are  the  Issues 
and/or  Limitations  Associated  with  These  Metrics? 

The  following  is  a  list  of  the  current  Contracting  metrics  that  NAVSUP  is 
gathering  and  presenting  to  the  Chief  of  the  Navy  Supply  Corps  on  a  monthly  basis  as 
part  of  the  Chiefs  overall  “dashboard”  (see  the  Balanced  Scorecard  presented  in  Chapter 
II): 

1 .  Number  of  Reverse  Auctions  conducted  and  saving  achieved. 

2.  Percentage  of  Performance  Based  Service  Contracts  (PBSC). 

3.  Percentage  of  work  Competed. 

4.  Contract  Closeout  and  the  number  of  over-aged  contracts. 

5.  Navy  and  Marine  Corps  (NMCI)  -  Meeting  Contract  Obligation 
Requirements. 

6.  Small  Business  -  Perfonnance  Evaluation  Summary. 

7.  Customer  Satisfaction. 

8.  Cycle  Time. 

9.  Procurement  Management  Review  (PMR)  reviews  as  they  relate  to 
Purchase  Card. 

FISC  Norfolk’s  metrics  are  located  in  Appendix  A.  As  was  stated  in  Chapter  III, 
there  are  significant  limitations  to  the  current  metrics.  In  Chapter  II,  we  found  “it  is 
easier  to  develop  lower  level  scorecards  that  link  together  if  the  metrics  for  the  CEO  have 
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been  defined  first”.  You  can  develop  scorecards  at  many  different  levels  of  the 
organization.  The  key  is  that  they  link  to  higher-level  metrics  within  the  command  or  to 
a  higher  echelon  command.  NAVSUP,  as  a  Systems  Command,  must  focus  on  strategic 
and  in  most  cases  ASN  driven  metrics.  FISC  Norfolk,  a  lower  echelon  is  interested  in  a 
different  set  of  metrics  (combination  of  strategic  and  operating  metrics).  That  being  said, 
all  lower  level  metrics  or  operating  metrics  should  feed  into  the  higher  level. 
Unfortunately,  NAVSUP ’s  metrics  appear  to  be  merely  counting  and  “putting  out  the 
fire”  type  metrics.  For  instance,  how  many  Performance  Based  Service  Contracts  have 
you  awarded?  How  many  overage  closeouts  have  you  performed?  What  is  your 
Competition  rate?  What  are  your  Small  Business  Statistics?  How  many  reverse  auctions 
have  you  conducted  and  what  are  the  savings?  How  many  PMRs  were  conducted  and 
how  many  received  an  unsatisfactory  rating?  One  can  argue  that  none  of  these  metrics 
measure  the  efficiency  of  a  contracting  office.  Some  may  allude  to  the  effectiveness  of 
the  contracting  office  (customer  satisfaction,  NMCI  Obligation  rates),  but  overall  it 
would  be  difficult  to  determine,  just  based  on  these  metrics,  if  your  contracting  function 
is  efficient  or  effective.  For  example,  a  monthly  counting  of  the  number  of  reverse 
auctions  and  performance  based  service  contracts,  when  both  are  meant  to  be  contracting 
tools  one  should  consider  when  determining  the  best  approach  to  meeting  a  customer’s 
requirements,  does  not  provide  you  with  an  insight  as  to  whether  these  tools  were  used 
appropriately  and  achieved  the  goal  of  satisfying  the  customers.  There  are  two  mistakes 
here. 

First,  establishing  reverse  auctions  as  a  dashboard  metric  for  the  Chief  of  the 
Naval  Supply  Systems  Command  does  not  seem  appropriate.  This  measure  does  not 
provide  an  accurate  barometer  as  to  whether  the  contracting  function  is  operating 
“efficiently  and  effectively  to  deliver  combat  capability  through  logistics”.  If  used  at  all, 
reverse  auctions  should  be  a  lower  level  metric. 

Second,  using  a  reverse  auction  goal  and  measuring  an  office  against  that  goal 

sends  the  wrong  message.  It  may  motivate  contracting  offices  to  fit  a  round  hole  into  a 

square  peg.  Reverse  auctions  may  be  an  excellent  tool  under  the  right  circumstances. 

The  key  is  to  ensure  that  your  contracting  officers  are  trained  and  know  the  value  of  all 

the  tools  available  to  them.  Performance  Based  Service  Contracts  (PBSCs),  Small 
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Business,  and  Contract  Closeout  fall  into  this  same  category,  with  a  major  difference, 
ASN  has  mandated  these  metrics.  ASN  requires  that  50%  of  all  eligible  service  contracts 
must  be  performance  based  by  FY  05.  Again,  similar  to  reverse  auction,  one  can  argue 
that  a  performance  based  service  contract  is  just  another  tool  for  a  contracting  officer  to 
consider  when  determining  the  best  way  to  satisfy  the  customer.  In  addition,  assigning  a 
performance  based  services  contract  goal  against  a  contracting  office  violates  one  of  the 
tenets  of  selecting  a  metric;  “the  metric  owner  has  the  ability  to  influence  the  metric”. 
While  contracting  offices  have  a  duty  to  train  requirements  generators  in  the  benefits  of 
performance-based  contracts,  they  do  not  define  the  actual  requirements.  Many  times 
the  PCOs  find  themselves  struggling  to  convert  a  non-performance  based  statement  of 
work  into  a  perfonnance  based  statement  of  work  while  still  satisfying  the  basic 
requirements  of  the  service  requested.  There  may  be  a  strong  argument  to  utilize  a 
performance  based  contract,  however,  as  GAO  pointed  out  (GAO  “Contract 
Management”:  Guidance  Needed  for  Using  Performance  Based  Service  Contracting), 
many  still  do  not  know  all  the  steps  required  in  defining  a  performance  based  service 
contract  and  more  guidance  is  needed  before  determining  the  actual  benefit  of  this  type  of 
contract. 

NAVSUP’s  one  metric  that  lends  itself  to  efficiency  is  the  Cycle  Time  Metric. 
However,  this  metric  also  has  some  problems.  Cycle  time  measures  the  percentage  of 
time  you  award  a  contract/order  in  less  than  20  days  for  acquisitions  less  than  or  equal  to 
$25,000  and  30  days  for  acquisitions  between  $25,000-$  100,000.  One  of  the  problems 
associated  with  this  metric  is  performance  motivation.  On  many  occasions,  the  customer 
does  not  require  a  contract/order  within  the  20  or  30-day  goal  (finding  issues,  required 
delivery  dates,  sending  in  a  requirement  60  days  before  a  contract  is  needed).  On  the 
other  hand,  some  customers  need  their  requirements  filled  immediately.  PCOs  must  be 
capable  of  prioritizing  workload  without  having  to  worry  about  meeting  some  arbitrary 
goal  that  does  not  provide  a  measure  for  efficiency  and  can  seriously  hinder  your 
effectiveness.  Most  customers  would  not  consider  you  too  effective  if  you  awarded  their 
requirement  in  19  days  when  they  needed  it  in  2  days.  This  kind  of  metric  may  have  a 
negative  impact  on  customer  relations  and  motivate  poor  business  decisions.  Not  many 
managers  would  be  happy  to  know  that  their  PCOs  would  be  willing  to  give  up  on  the 
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idea  of  negotiating  for  a  better  price  because  they  needed  to  get  an  award  done  within  the 
cycle  time  goals.  In  addition,  NAVSUP  did  not  consider  the  transaction  costs  associated 
with  gathering  and  managing  this  metric.  The  Standardized  Procurement  System  (SPS) 
had  to  be  modified  and  negotiators  were  required  to  track  the  reasons  as  to  why  they  did 
not  meet  the  cycle  time  goals  and  these  had  to  be  reported  monthly.  The  only  way  to 
track  the  reasons  is  manually  and  is  labor  intensive.  One  has  to  question  the  value  in 
gathering  this  metric  when  you  analyze  the  cost  of  gathering  it. 

All  of  NAVSUP’s  metrics  were  established  with  no  analysis  of  the  management 
information  systems  available  at  each  of  their  field  offices.  SPS  does  not  have  the 
capability  to  provide  all  of  the  metrics  required.  Also,  all  of  NAVSUP’s  metrics  are 
“after  the  fact”  or  past  metrics  and  does  not  look  at  Work-in-Process  (WIP)  or  any  future 
metrics.  They  utilize  PMRS  and  MOCAS  for  some  metrics  and  manual  counts  for  others. 
One  of  the  basic  tenets  of  a  good  scorecard  is  to  have  a  metric  that  looks  as  the  past 
present  and  future  in  order  to  understand  trends  and  motivate  future  performance. 

Unfortunately,  some  of  the  metrics  established  by  FISC  Norfolk  does  not  look  at 
efficiencies  or  effectiveness  because  some  of  the  contracting  offices  do  not  have  the 
ability  to  easily  gather  these  metrics.  The  researcher  is  providing  a  model  as  part  of  his 
recommendations.  The  model  proposed  is  assuming  that  all  offices  have  the  same 
management  infonnation  system.  This  is  a  required  element  of  the  model  because  it  will 
ensure  that  all  have  the  capability  of  measuring. 

Some  of  the  problems  with  FISC  Norfolk’s  metrics  are  the  complexity  of  the 

measures.  Many  individuals  the  researcher  interviewed  understood  the  importance  of 

gathering  a  specific  measure,  but  did  not  understand  how  they  were  being  graded  against 

that  metric.  For  instance,  on  page  1 1  of  Appendix  A,  FISC  Norfolk  provides  a  sample  of 

the  efficiency  index.  It  was  difficult  to  understand  where  the  figures  for  Data  and  Score 

came  from.  Most  understood  the  fact  that  you  would  have  different  weighting  assigned 

to  different  measures  as  some  are  more  important  than  others,  but  the  overall  metric’s 

scoring  sheet  was  sometimes  difficult  to  understand.  Also,  some  metrics  fall  into  the 

same  category  as  NAVSUP’s  in  that  they  measured  processes  that  FISC  Norfolk  had 

little  to  no  influence  over.  Specifically,  the  Portal/EMALL  metric  assumes  that  we  have 

the  ability  to  add  contractors  and  catalogs  to  the  EMALL.  While  it  is  true  that  we  may 
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recommend  and  encourage  contractors  to  have  their  catalogs  added  to  the  EMALL,  we  do 
not  control  the  total  process  and  many  times  it  takes  months  in  the  DOD  EMALL  process 
chain  to  actually  get  a  contractor/catalog  added  to  the  site.  With  regards  to  this  metric, 
FISC  Norfolk  should  concentrate  only  on  the  areas  that  they  can  influence  (#  of 
customers  profded)  and  disregard  the  measure  of  catalogs  and  Indefinite  Delivery  Type 
Contracts  (IDTCs)  added  to  the  site. 

Another  problem  with  the  current  metrics  is  the  Customer  Satisfaction  measure. 
This  measure  is  controlled  by  NAVSUP  and  requires  the  customers  to  access  NAVSUP’s 
website  and  complete  a  customer  satisfaction  survey.  Many  customers  have  expressed 
dissatisfaction  with  this  process  mainly  because  they  are  being  asked  on  a  quarterly  basis 
to  fill  out  a  satisfaction  survey.  In  addition,  there  are  many  different  customers 
associated  with  some  Commands  and  it  is  often  difficult  to  identify  what  specific 
customer  within  that  Command  needs  to  fill  out  the  survey.  Also,  on  some  occasions,  the 
sample  size  used  to  evaluate  customer  satisfaction  was  so  small  that  the  sample  could  not 
be  relied  on  to  provide  an  accurate  measure  of  the  effectiveness  of  customer  support. 

d.  What  Type  of  Metrics  Model  is  Required  in  Order  to  Measure  the 
Efficiency  and  Effectiveness  of  a  Navy  Field  Contracting  Office? 

As  stated  in  the  answer  to  the  primary  research  question,  a  balanced 
scorecard  metrics  model  (Appendix  B)  is  presented  as  a  method  for  measuring  the 
efficiency  and  effectiveness  of  a  Filed  Contracting  Office. 

D.  SUGGESTIONS  FOR  FURTHER  RESEARCH 

While  pursuing  this  thesis,  the  researcher  found  one  major  area  that  should  be 
further  researched: 

1.  Measuring  the  Complexity  of  an  Individual  Contract  Action 

Because  measuring  the  productivity  of  a  contracting  office  is  difficult,  the  Navy 
has  a  tendency  to  lump  together  all  of  the  different  types  of  contract  actions  generated  by 
a  contracting  office.  As  the  research  indicated,  the  Navy  seems  to  think  an  “1102  is  an 
1102”.  The  Navy  also  tends  to  think,  “An  action  is  an  action”.  The  researcher  knows 
from  his  experience  that  not  all  actions  are  the  same  and  some  are  much  more  complex 
that  others.  While  the  researcher’s  model  segregates  the  current  work-in-process  by 
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action  type,  he  does  not  measure  the  time  it  takes  to  complete  a  particular  action.  Further 
research  into  developing  a  time  in  motion  study  or  some  other  study  should  be  conducted. 
The  ultimate  measure  of  efficiency  in  a  contracting  office  can  only  be  attained  when  you 
have  a  full  understanding  of  the  type  of  work  that  is  performed  there.  This  type  of 
analysis  should  always  be  done  before  comparing  contracting  offices  against  each  other. 
The  researcher  recommends  gathering  a  team  consisting  of  an  independent  review 
official  and  Navy  field  contracting  office  managers  and  chartering  them  to  establish  a 
standard  in  which  each  contracting  actions  is  measured.  It  should  be  noted  that  NAVSUP 
attempted  to  do  this  in  1994,  but  without  an  independent  review  authority.  The  team 
disbanded  after  the  third  meeting  because  they  were  all  gaming  the  process  to  the  benefit 
of  their  specific  activity.  By  incorporating  an  independent  review  official,  you  may  be 
able  to  subvert  the  political  pressure  that  the  prior  team  encountered.  Once  this  is 
established,  the  Navy  will  have  a  better  understanding  of  the  productivity  and  efficiency 
of  a  field  contracting  office. 


83 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


84 


APPENDIX  A.  FISC  NORFOLK  METRICS 


naval  Supply  Systems  Command 


Fleet  &  Industrial  Supply  Center  Norfolk 

FISC  ACQUISITION  GROUP 

Metrics  Alignment  Initiative 


Managsmmt 

Guide 


85 


APPENDIX  A 


Metrics  Management  Guide 


Table  of  Contents 

Corporate  Metrics  Alignment . 2 

The  Process . 3 

Types  of  Metrics . 4 

Checklist  for  Evaluating  Coiporate  Metrics . 4 

Scorecard  Metrics . 6 

The  Scorecard . 6 

Key  Success  Factors . 6 

Initiative  Metrics . 7 

Reporting  Requirements . 10 

Reporting  F ormat . 10 

Key  Success  Factors . 10 

Initiative  Metrics . 12 

Notional  Scorecard . 18 


FISC  Norfolk  Acquisition  Executive 
1968  Gilbert  Street  •  Suite  600 
Norfolk  VA  23511-3392 
Phone  757.443-1601*  Fax  757.443.1605 


86 


Corporate  Metrics  Alignment 

Corporate  Metrics  are  intended  to  provide  the  Commanding  Officer  and  the 
Acquisition  executive  Board  an  overview  of  mission  accomplishment  and 
movement  toward  vision  achievement 

There  continues  to  be  increased  emphasis  on  designing  metrics  to  drive  improvement 
while  maintaining  oversight  of  complex  operations.  Therefore,  performance  metrics 
should  encourage  performance  improvement,  effectiveness,  efficiency,  and  appropriate 
levels  of  internal  controls.  Where  practical,  they  should  incorporate  "best  practices" 
related  to  the  performance  being  measured  and  cost/risk/benefit  analysis,  where  appropriate. 
This  section  discusses  the  principles  and  concepts  used  in  developing  effective  performance 
metrics  for  FISC  Norfolk  Acquisition  Group  operations.  The  underlying  principles  of  metrics 
used  within  FISC  Norfolk  are  based  upon  the  writings  of  Mark  Graham  Brown  is  his  books 
“Keeping  Score”  and  ‘Winning  Score”.  Additionally,  ‘The  Balanced  Scorecard -  Translating  Strategy  into 
Action”  model  of  David  Norton  and  Robert  Kaplan  will  provide  a  recommended  approach  to 
ensure  a  complete  set  of  metrics  is  promoted.  This  guide  assumes  reader  familiarity  with  these 
topics,  and  as  such  will  avoid  detail  review  of  their  principles. 


Naval  Supply  Systems  Command  has  promulgated  specific  guidelines  for  metrics  in: 

•  Customer  satisfaction 

•  Organizational  performance 

•  Workforce  excellence 


Additionally,  the  Acquisition  Group  Business  Plan  identifies  critical  initiatives.  The  key  elements 
of  the  performance  metrics  package  to  meet  these  initiatives  should  address  the  following: 

•  Attaining  Organizational  Alignment 

•  Improvements  in  quality,  cost  reduction  and/ or  avoidance 
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•  Meeting  customer  requirements 

•  Meeting  NAVSUP  requirements 

•  Identification  and  development  of  resource  requirements 


The  Process 

The  first  step  in  developing  performance  metrics  is  to  involve  the  people  who  are  responsible  for 
the  work  to  be  measured  because  they  are  the  most  knowledgeable  about  the  work.  Within  the 
Acquisition  Group,  these  people  are  identified  as  the  initiative  owners  within  the  Acquisition 
Group  Business  Plan.  Once  identified  and  involved,  they: 

•  Identify  critical  work  processes  and  customer  requirements. 

•  Identify  critical  results  desired  and  align  them  to  customer  requirements. 

•  Develop  measurements  for  the  critical  work  processes  or  critical  results. 

•  Establish  performance  goals,  standards,  or  benchmarks. 


The  establishment  of  performance  goals  can  best  be  specified  when  they  are  defined  within  three 
primary  levels: 

•  Strategic  Objectives:  Broad,  general  areas  of  review.  These  generally  reflect  the  highest 
goals  based  upon  mission  accomplishment  and  vision  achievement.  These  metrics  will 
comprise  the  corporate  level  metrics. 

•  Operational  Criteria:  Specific  areas  of  accomplishment  that  satisfy  major  divisions  of 
responsibility  within  a  function.  These  metrics  are  generally  mission  focused  and  form 
the  underpinning  of  strategic  metrics. 

•  Warning  Measures:  Metrics  designed  to  drive  improvement  and  characterize  progress  or 
problems  within  each  operation.  These  are  specific  quantifiable  goals  based  on  individual 
expected  work  outputs  of  critical  processes  or  contributory  functions. 


The  SMART  test  is  frequently  used  to  provide  a  quick  reference  to  determine  the  quality  of  a 
particular  performance  metric: 

S  =  Specific:  clear  and  focused  to  avoid  misinterpretation.  Should  include  measure  assumptions 
and  definitions  and  be  easily  interpreted. 
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M  =  Measurable:  can  be  quantified  and  compared  to  other  data.  It  should  allow  for  meaningful 
statistical  analysis.  Avoid  "yes/no"  measures  except  in  limited  cases,  such  as  start-up  or  systems- 
in-place  situations. 

A  =  Attainable:  achievable,  reasonable,  and  credible  under  conditions  expected. 

R  =  Realistic:  fits  into  the  organization's  constraints  and  is  cost-effective. 

T=  Timely:  doable  within  the  time  frame  given. 


Types  of  Metrics 

Quality  performance  metrics  allow  for  the  collection  of  meaningful  data  for  trending  and  analysis 
of  rate-of-change  over  time. 

Examples  are: 


•  Trending  against  known  standards:  the  standards  may  come  from  either  internal  or 
external  sources  and  may  include  benchmarks. 

•  Trending  with  standards  to  be  established:  usually  this  type  of  metric  is  used  in 
conjunction  with  establishing  a  baseline. 

•  Milestones  achieved. 


"Yes/no"  metrics  are  used  in  certain  situations,  usually  involving  establishing  trends,  baselines,  or 
targets,  or  in  start-up  cases.  Because  there  is  no  valid  calibration  of  the  level  of  performance  for 
this  type  of  measure,  they  should  be  used  sparingly. 

Checklist  for  Evaluating  Corporate  Metrics 

The  “scorecard”  concept  will  be  used  by  FISC  Norfolk  to  summarize  the  results  of  critical 
metrics.  The  following  checklist  from  “Winning  Score ”  wi  1 1  assist  in  evaluating  the  scorecard: 

•  A  reasonable  number  of  metrics  is  less  than  20 

•  Measures  are  balanced  against  past,  present  and  future  time  perspectives 

•  All  key  success  factors  and  business  initiatives  have  at  least  one  metric 

•  Weighting  will  be  used  to  highlight  critical  metrics 

•  Metrics  should  avoid  being  manipulated 

•  Indices  should  be  used  to  ease  data  review  and  aid  in  analysis 
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•  Measures,  and  initiatives  are  aligned  with  the  priorities  of  customers,  employees  and 
stakeholders 

•  Process  measures  correlate  to  outcome  measures 

•  Business  fundamentals  will  be  included  in  the  scorecard 
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Scorecard  Metrics 


The  Scorecard  is  designed  to  provide  a  summary  view  of  the  underlying 
corporate  metrics  to  Commanding  Officer  and  the  Acquisition  Executive 
Board. 

There  is  an  effort  to  streamline  the  FISC  Norfolk  metrics  brief.  Although  many 
departments  have  metrics  and  data  charts  that  should  be  maintained  at  the  departmental 
level,  many  do  not  rise  to  an  ESC  command  level  review.  To  pare-down  these  metrics,  a 
scorecard  has  been  designed  for  the  command  level.  Whenever  performance  as 
indicated  on  the  scorecard  warrants  more  detailed  review,  the  underlying  metrics  and  graphics  will 
be  used 


This  section  discusses  the  scorecard  and  its  associated  metrics  for  FISC  Norfolk  Acquisition 
Group  operations.  .It  will  guide  the  Group  in  data  collection  and  presentation  to  ensure  timely, 
consistent  data  collection  and  presentation 


The  Scorecard 

The  first  step  in  developing  the  scorecard  is  to  determine  the  performance  metrics  to  be 
displayed.  Within  the  Acquisition  Group,  these  metrics  are  those  provided  by  the  initiative  owners 
and  approved  by  the  Acquisition  Executive  Board. 

The  initiatives  are  aligned  with  the  Naval  Supply  Systems  Command  Strategic  Plan  Goals.  The 
strategic  metric,  objectives,  and  timing  associated  with  each  of  these  initiatives  will  form  the  basis 
for  the  Scorecard  metric. 

The  Scorecard  will  utilize  a  “Red/Yellow/Green”  format  to  summarize  performance. 
Determination  of  the  level  of  performance  and  allowable  variation  will  set  the  criteria  for 
appropriate  color  coding,  while  trending  observations  will  be  require  two  consecutive  movements 
in  the  same  direction. 

Key  Success  Factors 

The  Key  Success  Factor  metrics  are  indices  reflecting  both  mission  accomplishment  and  forward 
movement  on  vision  attainment.  The  standards  for  growth  metrics  are  typically  determined  by 
the  fiscal  month  of  the  year,  e.g.  in  December  we  should  have  accomplished  25%  of  the  goal;  or 
by  the  initiatives  plan  for  implementation.  Minimum  grades  or  Statistical  Process  Control  sets  the 
standards  for  process  metrics  as  appropriate.  Individual  metrics  are  not  coded  as 
“Red/Yellow/ Green”.  Instead,  the  indices  are  rated,  with  the  standard  set  by  the  summary  of  the 
individual  metric  within  the  respective  index.  “Green”  will  be  assigned  to  any  index  achieving  or 
within  five  percentage  points  of  that  standard,  “yellow”  will  be  assigned  to  any  index  scoring 
between  fifteen 
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and  five  percentage  points,  and  “red”  will  be  assigned  to  all  others.  All  initiative  owners  will 
report  their  metrics  to  ABM  to  support  monthly  reporting.  All  sites  will  provide  their  specific 
data,  such  as  %  of  training  hours  or  %  metrics,  to  initiative  owners  at  least  two  days  prior  to  the 
ABM  due  date  for  command  submission. 


•  Effectiveness  Index:  This  consists  of  three  weighted  individual  metrics. 


Metrics 

Initiative  Owner 

Frequency 

Standard 

Source 

#  of  Customers 
Profiled 

Marketing 

Monthly 

A  growth  metric  determined  by  the  annual 
plan. 

Manual  count 

Catalogues  and 

IDTCs  on  Portal 

Portal 

Monthly 

A  growth  metric  determined  by  the  annual 
plan 

Manual  count 

Customer 

Satisfaction  Index 

CRM 

Monthly 

A  process  metric.  The  standard  is  always 
95%  Highly  Satisfactory  or  above. 

NAVSUP  Database 

•  Efficiency  Index:  This  consists  of  four  weighted  individual  metrics. 


Metrics 

Initiative  Owner 

Frequency 

Standard 

Source 

Intranet  Hits 

Intranet  PM 

Monthly 

A  growth  metric  determined  by  the 
desired  goal  of  each  employee  accessing 
the  site  every  other  day. 

Website  Hit  Counter 

Customers  on  Portal 

Portal 

Monthly 

A  growth  metric  determined  by  the  annual 
plan 

Manual  count 

e-Transactions  Index 

Innovative, 

Streamlined 

Acquisition  Processes 
and  Tools 

Monthly 

A  process  metric.  The  standard  is  always 
92%.  The  metric  will  use  the  OSD  report, 
but  only  include  FISC-controlled 
processes  of  solicitations,  award  and 
closeouts.  In  final  calculations,  all 
subsidiary  metrics  associated  with  the 
awards  metric  will  be  counted  individually. 

OSD  Paperless  Metrics  Report 
from  ABM 

%  Paperless  Review 
Board 

COC  of  each  site 

Monthly 

A  growth  metric  determined  by  the 
desired  goal  of  achieving  95%  of  review 
boards  held  by  electronic  media. 

Manual  count  from  the  Review 
Board  log. 

•  Sustainment  Index:  This  consists  of  four  weighted  individual  metrics. 


Metrics 

Initiative  Owner 

Frequency 

Standard 

Source 

Series  Standards 
Profiled 

Personnel 

Development 

Monthly 

A  growth  metric  determined  by  the 
desired  goal  of  profiling  ten  critical 

Series/ specialties. 

Manual 

%  IDPs  Executed 

Personnel 

Development 

Monthly 

A  growth  metric  determined  by  number  of 
people  in  the  ten  critical 
series/  subspecialties 

Manual  count 

Training 

requirements 

resourced 

Personnel 

Development 

Monthly 

A  growth  metric  determined  by 
identification  of  resources  identified  for 
the  ten  critical  series/ subspecialties 

Manual  count  from  Access 
Training  Database 

Training  Hours  Met 

Each  Detachment 

Monthly 

A  growth  metric  determined  by  the 
desired  goal  of  achieving  100%  of  the  40- 
hour  requirement. 

HRO 

Initiative  Metrics 


Each  initiative  will  have  several  initiatives  associated  with  them.  The  scorecard  will  capture  at  least 
one  metric  for  them  The  standards  for  growth  metrics  are  typically  determined  by  the  fiscal 
month  of  the  year,  e.g.  in  December  we  should  have  accomplished  25%  of  the  goal;  or  by  the 
initiatives  plan  for  implementation.  Minimum  grades  or  Statistical  Process  Control  sets  the 
standards  for  process  metrics  as  appropriate.  “Red /Yellow/ Green”  indicators  are  assigned  to 
individual  metrics. 
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Monthly  metrics  will  be  reported  for  the  Acquisition  Group  as  a  whole.  Individual  sites  will 
report  quarterly  metrics,  with  the: 

•  First  month  of  each  quarter  reflecting  Philadelphia, 

•  Second  month  reflecting  Norfolk,  and 

•  Third  month  reflecting  Washington  metrics. 


Metrics 

Initiative  Owner 

Frequency 

Standard 

Source 

Customer 

Satisfaction 

Survey 

CRM 

Quarterly 

A  process  metric  with  standards  set  by  NAVSUP.  Green  equals  attainment 
of  NAVSUP  goal  of  highly  satisfactory  rating.  Yellow  equals  a  grade  within 
five  percentage  points  of  that  standard,  currently  set  at  90% 

NAVSUP 

Database 

Portal 

Customer 
Market  Share 

Product 

Development/ 

Portal 

Development 
and  Execution 

Monthly 

A  growth  metric  determined  by  dollar  value  of  sales  on  the  portal  divided  by 
the  total  sales  of  those  same  customers.  Green  will  equate  to  within  90%  of 
plan,  yellow  to  within  80%  of  plan.  The  plan  will  be  established  as  the 
projected  annual  sales  of  all  portal  customers,  to  suppliers  that  are  on  the 
portal..  Annual  sales  will  be  projected  based  upon  the  starting  month  of  the 
customer  portal  usage. 

IBM  for  Portal 
Sales,  Dynamic 
Reporting  for 
Total  Sales 

% 

Performanced 

Based 

Contracts 

Innovative, 

Streamlined 

Acquisition 

Processes 

Quarterly 

A  growth  metric  with  the  baseline  set  for  each  site  based  upon  previous  FY 
results.  Green  will  be  reported  if  the  score  exceeds  5%  above  the  prior  year, 
yellow  if  it  is  within  5%  plus  or  minus  of  the  prior  year. 

PMRS  reports 
from  ABM 
where  B1D= 
Yes/Total 
Contracts 
Awarded 

Reverse 

Auctions 

Innovative, 

Streamlined 

Acquisition 

Processes 

Monthly 

A  process  metric  determined  by  the  number  of  planned  auctions  compared 
to  the  NAVSUP  goal.  Green  equals  planned  attainment  of  the  goal.  Yellow 
equals  planned  auctions  within  20%  of  the  goal. 

Manual 

Self- 

Assessment 

Findings 

Self-Assessment 

Program 

Management 

Quarterly 

A  compilation  process  metric  combining  PMR,  SAP,  Large  Procurement 
and  Special  Interest  Item  QA  plan  scores.  Any  red  score  or  two  yellow 
scores  for  any  of  these  categories  will  equal  a  red  for  the  metric.  Any  yellow 
score  in  any  category  will  result  in  an  overall  yellow.  Green  will  be  assigned 
if  all  areas  score  green. 

Within  PMR,  total  audits  within  90%  of  schedule  will  be  green,  while  within 
85%  will  be  yellow. 

Within  SAP,  excess  error  rates  in  one  area  will  lead  to  a  yellow  rating,  while 
two  will  lead  to  a  red 

Within  Large  Procurement,  excess  error  rates  in  two  areas  will  lead  to  a 
yellow,  while  three  will  lead  to  a  red. 

Within  Special  Interest,  error  rates  greater  than  10%  will  equal  a  yellow,  and 
greater  than  20%  will  equal  a  red 

FISC  Norfolk 
Det 

Philadelphia 

QAPM 

Database 

Small 

Business 

Small  Business 
Managers 

Quarterly 

A  process  metric  with  standards  set  for  each  category.  Failure  to  achieve 
two  categories  will  equate  to  a  yellow,  while  failure  in  three  will  equate  to  a 
red 

PMRS 

Overage 

Contract 

Closeout 

Contract 

Closeout 

Monthly 

A  process  metric  with  green  achieved  when  monthly  closeouts  equal  or 
exceed  a  standard  set  at  the  total  overage  on  hand  divided  by  12.  Yellow 
will  be  assigned  if  closeouts  are  within  10%  of  that  standard.  All  sites  must 
reflect  green  for  an  overall  grade  of  green.  Any  site  with  a  red  will  result  in 
a  grade  of  red. 

DRID  53  and 

Workload 

Staffing 

Reports 

SPS  Upgrade 
Effectiveness 

Procurement 

Systems 

Management 

Quarterly 

TBD 

TBD 

%  of 

Customer 
Requirements 
met  via  the 
Portal 

Portal 

Development 
and  Execution 

Monthly 

A  process  metric  determined  by  the  number  of  transactions  on  the  portal 
divided  by  the  total  number  of  customer  Purchase  Card  transactions.  The 
standard  will  be  set:  as  the  percentage  of  transactions  expected  from  their 
catalogue  and  contract  vendors  on  the  portal  as  determined  by  the  %  those 
suppliers  achieved  in  the  past  fiscal  year.  A  score  of  90%  will  be  green, 
while  less  than  80%  will  be  red. 

IBM  for  Portal 
Actions, 
Dynamic 
Reporting  for 
Total  Actions 

%  of  FISC 

Markets 

Sourced 

Market 

Managers 

Quarterly 

A  growth  metric  determined  after  critical  assessment  of  potential  for 
requirements  consolidation  through  contracts.  The  annual  plan  to  analyze 
and  award  contracts  by  FSC  will  set  the  standard 

Manual 
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NMCI  Order 

Fulfillment 

Rates 

NMCI 

Quarterly 

A  process  metric  determined  by  comparing  actual  orders  executed  to  the 
program  manager’s  plan.  Green  will  indicate  the  orders  executed  are 
running  to  plan.  Yellow  will  indicate  0%-10%  behind  plan. 

Manual 

Employee 

Satisfaction 

HRSC 

Annual 

A  process  metric  provided  through  command  assessment 

HRSC  Survey 

Alignment 

Metrics 

Alignment 

Quarterly 

A  process  metric  reflecting  results  of  the  quarterly  alignment  survey. 

Standards  are  set  for  absolute  and  relative  scores: 

(1)  Absolute:  If  the  lowest  score  is  below  35  it  will  be  yellow,  if  below  25 
it  will  be  red 

(2)  Relative:  If  the  difference  between  the  highest  and  lowest  score 
exceeds  five,  this  will  be  yellow;  if  it  exceeds  ten,  this  will  be  red 

If  both  parameters  are  either  red,  or  if  there  is  one  yellow  and  one  red,  the 
alignment  metric  is  red.  If  only  one  parameter  is  red,  the  metric  is 
yellow.  If  only  parameter  is  yellow,  the  metric  is  green. 

Alignment 

Survey 

Database 

Intern 

Vacancies 

ABM 

Monthly 

A  growth  metric  determined  by  the  number  of  intern  billets  without  fills  or 
selections.  Vacancies  in  excess  of  10%-20%  of  authorization  will  be  yellow, 
greater  than  20%  will  be  red.  However,  after  April  of  each  fiscal  year, 
yellow  will  be  0%-10%  and  red  will  be  greater  than  10%. 

Manual 
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Reporting  Requirements 

The  Scorecard  requires  timely  reporting  of  accurate  data  in  order  to  he  a 
useful  tool. 

The  monthly  Scorecard’s  development  is  a  team  effort.  Acquisition  Group  metric  owners 
must  contribute  their  data  in  a  timely  and  accurate  manner.  The  ABM  division  is 
responsible  for  the  compilation  of  data  for  further  submission  to  the  FISC  Norfolk 
metrics  manager.  These  metrics  are  usually  due  near  the  end  of  the  first  full  week  of  each 
month,  although  a  specific  schedule  is  promulgated  with  exact  due  dates.  It  is  critical  that  each 
metric  owner  become  familiar  with  the  date  reporting  format  and  schedule  to  ensure  the 
Acquisition  Group  Scorecard  is  as  accurate  as  possible  for  the  monthly  review  and  subsequent 
posting  to  the  Intranet. 

Reporting  Format 

The  following  worksheet  will  be  used  to  provide  standardized  input  to  the  ABM.  Owners  may 
submit  their  input  via  e-mail;  however,  they  should  ensure  the  data  is  reported  via  the  format  of 
this  section. 

Key  Success  Factors 

1.  Effectiveness  Index::  This  consists  of  three  weighted  individual  metrics.  Each  site  will 
provide  input  from  their  offices  on  customer  profiling  and  new  catalogues  to  the 
Acquisition  Group  initiative  lead.  The  CRM  lead  will  obtain  data  from  the  NAVSUP 
database  on  the  customer  satisfaction  index  for  all  sites.  It  will  consist  of  the  %  of  all  data 
scores  received  for  all  elements  that  are  rated  highly  successful  or  better. 


Month 

Site 

Comments 

Metrics 

Weighting 

Plan 

Actual 

Data 

Score 

Weight 

Data 

Score 

Weight 

a.  #  of  Customers 

Profiled: 

.30 

b.  Catalogues  and  IDTCs 
on  Portal: 

.30 

c.  Customer  Satisfaction 
Index: 

.40 

Index 

Total 

Total 

Color 

Trend 
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2.  Efficiency  Index::  This  consists  of  four  weighted  individual  metrics.  Detachment  Philadelphia 
will  provide  intranet  data  direcdy  to  ABM.  Each  site  will  provide  input  from  their  offices  on 
customer  portal  to  the  initiative  lead.  Each  site  will  forward  their  e-transactions  data  from  the 
OSD  Paperless  Metrics  Report  to  ABM.  The  ABM  will  calculate  the  final  number  based  upon 
the  total  of  the  actions  in  the  following  categories: 

•  Solicitations  issued 

•  Awards  and  modifications  issued  to: 

a.  Files 

b.  Originator 

c.  ACO 

d.  DFAS  for  payment 

e.  DFAS  for  accounting 

f.  Contractor 

•  Contract  Closeouts 

Each  site  will  review  their  CRB  log  to  provide  the  data  for  the  number  of  paperless  and  total  CRBs. 


Month 

Site 

Comments 

Metrics 

Weighting 

Plan 

Actual 

Data 

Score 

Weight 

Data 

Score 

Weight 

a.  #  of  Intranet  Hits 

.25 

b.  Customers  on  Portal 

.25 

c.  e-Transactions  Index 

.40 

95% 

80 

32 

d.  %  Paperless  Review 

Boards 

.10 

75% 

80 

8 

Index 

Total 

Total 

Color 

Trend 

3.  Sustainment  Index:  This  consists  of  four  weighted  individual  metrics.  The  Personnel 
Recruitment,  Development  and  Retention  lead  will  identify  how  many  series  have  been 
profiled.  Each  site  will  report  how  many  IDPs  are  due/have  been  reviewed  and  executed 
based  upon  profiled  series  standards.  They  will  also  report  how  many  of  the  training 
requirements  identified  in  the  completed  IDPs  have  been  sourced  and  resources  identified. 
Each  site  will  provide  input  from  their  offices  to  ABM  on  the  %  of  personnel  requiring  and 
completing  the  40-hour  training  requirement. 


Month 

Site 

Comments 

Metrics 

Weighting 

Plan 

Actual 

Data 

Score 

Weight 

Data 

Score 

Weight 

a.  Series  Standards  Profiled 

.30 

b.  %  IDPS  executed 

.30 

96 


c.  Training  Requirements 
Resourced 

.10 

d.  Training  Hours  Met 

.30 

Index 

Total 

Total 

Color 

Trend 

Initiative  Metrics 

4.  Customer  Satisfaction  Survey: 


Quarter 

Site 

Comments 

Quarter 

(Q/FY) 

Qty 

Unsat. 

Marginal 

Satisfactory 

Highly  Sat. 

Superior 

1: 

2: 

3: 

Color: 

Trend: 

5.  Portal  Customer  Market  Share: 


Month 

Site 

Comments 

Customers 

Months 
on  Portal 

Annual  Plan 

Actual  YTD  Performance 

Portal  Sales 

Portal  Sales 

Est. 

Plan  YTD 

Portal  Sales 

FISC  Norfolk 

579,026 

NNSY 

249,396 

SIMA 

79,792 

PWC 

91,044 

FOSSAC 

0 

NAVTRANS 

0 

DET  WASH 

25,000 

DET  PHIL 

3,000 

NAVICP 

100,000 

NAVSUP 

0 

NDW 

25,000 

Total 

1,152,258 

0 

0 

Percentage 

IZero  Divic 

e  Color 

Trend 
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6.  %  Performanced  Based  Contracts 


Month 

Site 

Comments 

Philadelphia 

Norfolk 

Washington 

Plan 

a.  FY01  % 

b.  x  10%  Growth 

Actual 

a.  Performanced  Based 
Contracts 

b.  Total  Contracts 

c.  Percentage 

Difference  to  Plan 

Color 

Trend 

7.  Reverse  Auctions: 


Month 

Site 

Comments 

Goal 

10 

Actual 

4 

Item 

Site 

Date 

Status 

(Planned/ Complete) 

Estimated  ^/Savings 
$ 

Color 

Trend 
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8.  Self  Assessment  Findings 


a.  PMR 


Month 

Site  Norfolk  &  Charleston 

Comments 

Actions 

Scheduled 

Completed 

% 

Desk  Audits 

PMR  and  Site  Audits 

Overall  Color 

Trend 

b.  QA  Plan 


Quarter 

Site 

Comments 

SP1 

SP2 

SP3 

SP4 

SP5 

Color 

SI1 

Color 

LC1 

LC2 

LC3 

LC4 

LC5 

LC6 

LC7 

Color 

Overall 

Color 

Trend 

9.  Small  Business: 


Quarter 

Site 

Comments 

SB 

SDB 

WOSB 

HZSB 

Vet 

HBC 

Month 

Goal 

34.4 

2.5 

5.0 

2.5 

3.0 

- 

Actual 

Overall 

Color 

Trend 
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10.  Overage  Contract  Closeout 


Month 

Site 

Comments 

Philadelphia 

Norfolk 

Washington 

NNSY 

On  Hand 

On  Hand/ 12 

Overage 

Completed 

DRID  53 

Difference  to  Plan 

Color 

Overall 

Color 

Trend 

1 1 .  SPS  Upgrade  Effectiveness  Quarterly  TBD  TBD 


12.  %  of  Customer  Requirements  met  via  the  Portal 


Month 

Site 

Comments 

Customers 

Months 

on 

Portal 

Annual  Plan 

Actual  YTD  Perfomance 

PC 

Actions 

Portal 

Action 

Est. 

%  of 
Est. 

Plan  YTD 

Portal 

Actions 

% 

FISC  Norfolk 

NNSY 

SIMA 

PWC 

FOSSAC 

NAVTRANS 

DET  WASH 

DET  PHIL 

NAVICP 

NAVSUP 

NDW 

Total 

0 

0 

Percentage 

Color 

Trend 
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13.  %  of  FISC  Markets  Sourced 


Month 

Site 

Comments 

Market 

#  ofFSCs 

Plan 

Actual 

Color 

Trend 

14.  NMCI  Order  Fulfillment  Rates 


Quarter 

Site 

Comments 

Increment 

Orders 

Deobligation  Mods 

1.0 

Planned 

Actual 

1.5 

Planned 

Actual 

2.0 

Planned 

Actual 

2.5 

Planned 

Actual 

Color 

Trend 
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15.  Employee  Satisfaction 


16.  Alignment 


Quarter 

Site 

Comments 

Strategy 

People 

Customer 

Process 

Score 

Lowest  Score 

Color 

Biggest 

Difference 

Color 

Color 

Trend 

17.  Intern  Vacancies 


Month 

Site 

Comments 

Billets 

Offers 

Filled 

Vacancies 

Color 

Trend 
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APPENDIX  B.  METRICS 


APPENDIX  B 


PROPOSED  MODEL 
FOR  MEASURING 
EFFICIENCY  AND 
EFFECTIVENESS  OF  A 
FIELD  CONTRACTING 
OFFICE 


BALANCED  SCORECARD 

STRATEGIC  METRICS 

OPERATIONAL  METRICS 

CUSTOMER  METRICS 

EMPLOYEE  METRICS 


2 
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COMPARISION  OF  2  FISCAL  YEARS: 
ACTIONS  OVER  25K  ACTIONS 


Source:  PMRS  Database  DF62N 

Does  not  include  actions  where  total  dollars  are  “0” 
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COMPARISION  OF  2  FISCAL  YEARS: 
DOLLARS  OVER  25K  ACTIONS 


$M 


Source:  PMRS  Database  DF62N 


LARGE  PURCHASE 
TOP  10  SUPPLIERS  BY  DOLLARS 


SUPPLIER 

FSC  DESCRIPTION 

TOTAL 

DOLLARS 

Electronic  Data  Systems  Corp 

Other  ADP  &  Telecommunication  Svcs 

$60,422,537 

Dell  Marketing  LP 

ADP  Central  Processing  Unit 

$44,871,014 

Innovative  Logistics  Technique 

Logistics  Support  Services 

$33,477,534 

General  Dynamics  Gov  Sy 

RDTE/Other  Defense  Adv  Tech  Dev 

$32,540,992 

AMSEC  LLC 

Maint  &  Repair  of  Eq/Miscellaneous  Equip 

$25,005,445 

Goodwill  Industries  of  SE  Wl 

Other  ADP  &  Telecom  Svcs 

$23,520,814 

Booz  Allen  &  Hamilton,  Inc. 

Tuition,  Registration  &  Membership  Fees 

$21,854,247 

Mantech  Systems  Engineering  Co. 

Other  ADP  &  Telecommunications  Svcs 

$20,964,620 

San  Diego  City  College 

Training/Curriculum  Development 

$19,485,168 

Lockheed  Martin  Services  In.c 

Training/Curriculum  Development 

$18,755,870 
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LARGE  PURCHASE 
TOTAL  ACT/OBS  FOR 
SUPPLIES/SERVICES 


Actions 

Obligations 

Supplies 

163 

$59,858,299 

Services 

1424 

$650,497,165 

LARGE  PURCHASE 
TOP  10  FSC’S  BY  ACTIONS 
(SERVICES) 


Other  ADP  &  Telecommunication  Services  (D399) 

203 

Logistic  Support  Services  (R706) 

173 

Tech  Rep  Svcs/Aircraft  Gnd  Handling  Equip  (L017) 

97 

Other  Medical  Services  (Q999) 

64 

Maint  &  Repair  of  Eq/Misc  Equipment  (J099) 

61 

Engineering  Technical  Services  (R425) 

57 

RDTE/Other  Defense-Adv  Tech  Dev  (AD93) 

55 

Training/Curriculum  Development  (U008) 

44 

Maint  &  Repair  of  Eq/Ships-Sml  Craft-Docks  (J019) 

40 

Tech  Rep  Svcs/Aircraft  Comps.  &  Accys  (L016) 

37 
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COMPETITION 

A  monthly  update  of  the  actual  competition  rate  versus  the  goal 

% 

80 

60 

40 

20 

1 

OCT 

NOV 

DEC 

JAN 

FEB 

MAR 

APR 

MAY 

JUN 

JUL 

AUG 

SEP 

MONTHLY 

39.2 

96.9 

98.1 

88.5 

88.1 

61.2 

78.9 

84 

80.6 

□  CUMULATIVE 

39.2 

52.9 

62.6 

78.9 

81.1 

78.4 

78.7 

80.8 

81.9 

—  OBS  GOAL 

82 

82 

82 

82 

82 

82 

82 

82 

82 

82 

82 

82 

Goal  82% 

9 

E-BUSINESS 
(Paperless  Metrics) 
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ANTICIPATED  FUTURE 
OBLIGATIONS 

Description 

Customer 

Award 

Date 

Obligations 

FY02  FY03  FY04  FY05  FY06  FY07 

i 

ETS  for  C4I 

SPAWAR  $86M 

Aug  02 

$  3M 

$15M 

$  16M 

$  17M 

$18M 

$17M 

2 

BPR/ABC 

SUPPORT  SVCS 

JFCOM  $25M 

Jun  02 

$3.0M 

$5.0M 

$5.0M 

$6.0M 

$6.0M 

3 

ETS 

NAVSHIPSO  $160M 

Sep  02 

$11. 0M 

$32. 0M 

$32. 0M 

$32. 0M 

$32M 

$21 M 

4 

CA  Food,  Mat  Magt, 
Custodial 

BUMED  CAL  REG 
$50M 

Apr  03 

$5M 

$10M 

$10M 

$10M 

$10M 

5 

GTMO  Telecom. 

NCTAMSLANT 
$29. 9M 

Apr  02 

$  3.3M 

$  3.4M 

$3.5M 

$  3.6M 

$  3.7M 

6 

NTCSS  Fit  Support 
Svcs 

SPAWARSYSCEN 

$154M 

Apr  02 

$29. 5M 

$30. 7M 

$32. 0M 

$33. 4M 

$14. 2M 

7 

Curriculum 

Development 

NETPDTC  $29M 

May  02 

$  5.0M 

$  5.9M 

$  5.9M 

$  5.9M 

$  5.9M 

8 

Navy  College 
Learning  Prog. 

NETPDTC  $49 M 

Jul  02 

$2.0M 

$  9.8M 

$  9.8M 

$  9.8M 

$  9.8M 

$7.0M 

9 

Integrated  Supply 
Support 

PNS  Great  Lakes 

$25M 

Sep  02 

$5M 

$5M 

$5M 

$5M 

$5M 

10 

CA  Admin 

SUPSHIPS  $100M 

Aug  02 

$20M 

$20M 

$20M 

$20M 

$20l\h> 

CONTRACT  CLOSE-OUT 
STATUS 
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CONTRACT  CLOSE-OUT 
DOLLARS  DEOBLIGATED 


TOTAL 

F/Month 

Year-To-Date 

1990 

0 

582 

1991 

0 

0 

1992 

0 

177,179 

1993 

0 

0 

1994 

0 

16,010 

1995 

0 

24,509 

1996 

0 

0 

1997 

0 

0 

1998 

0 

168,898 

1999 

22,794 

258,720 

2000 

2,024 

25,683 

TOTAL 

0 

671,581 

MONTHLY  CLOSEOUT 
METRICS 


Data 

Category 

FISC 

(GTE  $25K) 

NAVSUP 
(GTE  $100K) 

ASN 

Website 

Beg  Balance 
Eligible 

4129 

1561 

3269 

New  Eligible  this 
Period 

120 

61 

205* ** 

Contracts  Closed 
this  Period 

336 

96 

96 

Ending  Balance 
Eligible 

3913 

1526 

3378* 

Overage  Ending 
Balance 

1569 

538 

N/A 

*  205  number  includes  contracts  imported  for  months  of  March,  April  and  May  2002 
(3  months  added  to  ASN  website  on  1 1  June  2002) 

**  ASN  website  still  to  be  updated  to  reflect  completed  closeouts 
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SMALL  BUSINESS 
GOALS/A  CHI EVEM ENTS 


FISC  Norfolk 
$ 

Det  Phila. 

$ 

FISC 

Norfolk 

Target 

FISC 

Norfolk 

Actuals 

Det 

Phila. 

Actuals 

Total  to  U.S.  Firms 

$953,909,039 

$514,736,321 

Small  Business 

$342,156,226 

$133,504,458 

34.4% 

35.9% 

25.9% 

SDV 

$580,793 

$245,610 

3.0% 

0.1% 

0.05% 

Small  Disadvantaged 
Business 

$140,693,354 

$77,374,168 

11.2% 

14.8% 

15.0% 

Women-Owned  Small 
Business 

$63,282,690 

$45,871,953 

3.8% 

6.6% 

8.9% 

HUB  Zone 

$15,265,341 

$1,718,730 

2.5% 

1.6% 

0.3% 

HBCU/MI 

$18,462,301 

$18,462,301 

22.5% 

77.1% 

77.8% 

BUSINESS  DEVELOPMENT 


Customer 

Descrmtion  of  Initiative 

Status 

OSD/C3I  PIMS 
International 

Outreach 

Partnership  for  Peace 
Information  Mgmt 

Svstems  Svcs  Support 

Contractual  planning  mtg  to  be 
scheduled  for  support  of  PIMS 
program. 

JFCOM  (J8) 

Technical  Support  Svcs 

Met  5/6;  expect  SOW  NLT  6/30. 

CNRNE 

Husbanding  Contracts 

RFP  issued  6/5/02. 

Naval  War  College 

War  Gaming  Support 

Svcs 

BAA  drafted.  Pre-Solicitation 
Conference  6/25/02. 

Joint  Forces 
Intelligence 

Command 

General  Contracting 
Support 

J4  advised  that  JFIC  will  be 
forwarding  some  requirements  to 
us. 

16 
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02P1 

MAJOR  PROGRAM 
SUPPORT 

• 

ERP 

• 

Inactive  Ships 

• 

NEMMI 

• 

CNET 

• 

JFCOM 

• 

FMS 

• 

Travel  Management 

• 

Shipboard  Copier 

• 

A-76 

• 

Acquisition  Reform  Office 

• 

NAVICP  FMS  Logistic  Support 

• 

Shop  Towels 

• 

Task  Force  Excel 

17 

OPERATIONAL 

METRICS 


18 
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CYCLE  TIME  METRICS 
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COMPETITIVE  SOURCING 


A-76  Metrics  * 


Cost 

Comparison 
Method  Used 

In  Process 

Completed 

Gov’t 

Retained 

Full  Study 

17 

23 

21 

65  and  Under 

3 

31 

31 

10  and  Under 

10 

47 

36 

Direct 

Conversion 

1 

16 

N/A 

*  Plus  33  Cancellations  of  Studies 


STREAMLINED  A-76 
PROCESS 


A-76  Metrics  * 


Process 

Studies 

Completed 

Average 

PALT 

PALT 

Range 

Full  Study 

23 

11.74months 

6-13  months 

65  and  Under 

31 

3.45  months 

1-6  months 

10  and  Under 

47 

3.04  months 

1-6  months 

22 
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WORK-IN-PROCESS  PROFILE 


WORK-IN-PROCESS  PROFILE 
ACQUISITION  STRATEGY 


TOTAL  FOLDERS  -  281 


24 
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WORK-IN-PROCESS  PROFILE 
CONTRACT  TYPE 


NEW  PROCUREMENTS  SUMMARY 
90  FOLDERS 


Measurement 

Competitive 

Sole 

Source 

Competitive 

Cl 

Sole 

Source 

Cl 

Total 

Unsolicited 

Phase 

PR  rec’t  date 
thru  sol.  Issue 
date 

30 

16 

4 

1 

51 

Solicitation 

Phase 

From  sol.  issued 
date  actual 
closing  date 

6 

4 

3 

0 

13 

Tech  Eval 
Phase 

After  sol.  closes 
&TE  is 

reauested 

7 

0 

1 

0 

8 

Negotiation 

Phase 

After  sol.  closes 

4 

12 

2 

0 

18 

Total 

471 

32 

10 

1 

90 

26 
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UNSOLICITED  PHASE 
jS^ROCUREMENTS^ 


Green 


Stratification 

O  0  -  30  days 
O  31-45  days 
•  Over  45  days 


Yellow 


Stratification 
0  0-15  days 
O  16-30  days 
#  Over  30  days 


SOLE  SOURCE  (16) 

Red 
6 


SOLE  SOURCE  Cl  (1) 


COMPETITIVE  (6) 


SOLICITATION  PHASE 
(13  PROCUREMENTS) 


COMPETITIVE  Cl  (3) 


Stratification 

O  0  -  30  days 
O  31-45  days 
#  Over  45  days 


Red 

1 


SOLE  SOURCE  Cl  (0) 


Stratification 
O  0-15  days 
O  16-30  days 
•  Over  30  days 


28 
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NEGOTIATION  PHASE 
(18  PROCUREMENTS) 


WITH 

TECH  EVAL 


Stratification 

O  0  -  30  days 
O  31-45  days 
#  Over  45  days 


SOLE  SOURCE  (12) 


WITHOUT 
TECH  EVAL 


Stratification 
0  0-15  days 
O  16-30  days 
•  Over  30  days 


SOLE  SOURCE  (0) 


30 
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DELIVERY  ORDERS  (74) 


O  0  -  15  days  #  Over  15  days 

TYPE  I  DOs  (14) 


TYPE  II  DOs  (15) 


O  0-25  days  O  26  -  45  days  •  Over  45  days 


O  0-25  days  O  26  -  45  days  •  Over  45  days 

31 


118 


Large  Purchase 

SaMMMumaiSi/ulum 


Metric 

Rating 

Standard 

Actual 

LC1  Pre-Award  Market  Research/AR 

V 

0% 

5% 

LC2  Pre-Award  RFP 

V 

0% 

15% 

LC3  Pre-Award  J&As 

V 

0% 

0% 

LC4  Award  Source  Selection 

s 

0% 

5% 

LC5  Award  Business  Clearances 

0% 

30% 

LC6  Post-Award  Closeout 

V 

0% 

16% 

LC7  Post-Award  File  Maintenance 

s 

0% 

DNR 

Simplified  Acquisition  Self- 
Assessment  Findings 


Metric 

Rating 

Standard 

Actual 

SP1  Pre-Award  Screening 

10% 

35% 

SP2  Pre-Award  Competition 

10% 

30% 

SP3  Pre-Award  Small  Business 

-/ 

15% 

16% 

SP4  Post-Award  Fair  &  Reasonable 

V 

10% 

40% 

SP5  Post-Award  Use  of  FSS 

10% 

25% 

SI1  Special  Interest  Reviews 

V 

10% 

5% 

34 
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Self  Assessment 
Recommendations 


•  Philadelphia 

-  All  Team  Leaders  are  responsible  for  signing  all  SAP  Actions 
Weekly  training  sessions  on  SAP  will  be  conducted  starting  Oct  01 
for  1105s  and  Team  Leaders 

-  Modify  the  GSA  Template  Memo 

-  Establish  SAP  Toolkit  and  post  on  Intranet 

-  Team  Leaders  and  negotiators  must  ensure  all  documentation  and 
chronology  of  events  are  in  file 


CUSTOMER  METRICS 


36 
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LARGE  PURCHASE 
TOP  10  CUSTOMERS  BY  ACTIONS 


Customer/UIC 

Total 

Actions 

NAVAL  AIR  SYSTEMS  COMMAND  HEADQUARTERS  (N00019) 

107 

NAVAL  EDUCATION  &  TRNG  PROF  MGMT  SUP  ACT  (N68322) 

99 

NAVAL  SHIPYARD,  NORFOLK  (N00181) 

96 

SPAWARS  (N66001) 

75 

FLEET  TECHNICAL  SUPPORT  CENTER  ATLANTIC  (N65912) 

70 

NAVAL  SEA  LOGISTICS  CENTER  CODE  134  (N65538) 

62 

JOINT  WARFIGHTING  EXP  BATTLE  LAB  JWEBL  (N3188B) 

60 

NAVICP  (N00391) 

51 

JTASC  (N39792) 

47 

NAVAL  AIR  TECH  DATA  &  ENGR  SVC  COMMAND,  NAS  NORTH  ISLAND 

41 

JAN  02  Customers  Provided.  PMRS  generates  inaccurate  FY02  data  due  to  DD350  block  changes  (from  Section  G  to  Block  E4). 
Working  on  solution. 
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LARGE  PURCHASE 
TOP  10  CUSTOMERS  BY  DOLLARS 


Customer/UIC 

Total  Dollars 

NAVSUPSYSCOM  HQ  (N00023) 

$51,909,654 

NAVAL  INVENTORY  CONTROL  POINT  (N00391) 

$50,168,532 

NAVAL  SHIPYARD,  NORFOLK  (N00181) 

$44,906,846 

JOINT  WARFIGHTING  EXP  BATTLE  LAB  JWEBL(N3188B) 

$36,439,804 

COMMANDER  IN  CHIEF  PACIFIC  FLEET,  NAVBASE  (N00070) 

$34,428,259 

NAVAL  EDUCATION  &  TRNG  PROF  MGMT  SUP  ACT  (N68322) 

$31,291,146 

NAVAIRSYSCOM  HQ  (N00019) 

$25,798,773 

SPAWAR  CHESAPEAKE  (N68561) 

$24,466,863 

SERVICE  SCHOOL  COMMAND,  NTC  (N0580A) 

$21,712,997 

SPAWAR  CENTER  (N66001) 

$20,448,150 

JAN  02  Customers  Provided.  PMRS  generates  inaccurate  FY02  data  due  to  DD350  block  changes  (from  Section  G  to  Block  E4). 
Working  on  solution. 
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WHAT  DO  WE  BUY  FOR  OUR  TOP  TEN 
CUSTOMERS 


CUSTOMER 

FSC 

#  Actions 

NAVAIR-  N00019 

R706 

81 

7690 

8 

AC15 

7 

1680 

2 

5998 

2 

NATPDTC  -  N68322 

D399 

57 

U008 

26 

U099 

11 

D301 

2 

R699 

2 

WHAT  DO  WE  BUY  FOR  OUR  TOP  TEN 
CUSTOMERS 


CUSTOMER 

FSC 

#  ACTIONS 

NAVAL  SHIPYARD 

NORFOLK  -  N00181 

J099 

52 

R425 

11 

D399 

6 

R421 

4 

2010 

3 

FLEET  TECHNICAL 

SUPPORT  CENTER 

ATLANTIC  -  N6591 2 

L017 

70 

40 
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WHAT  DO  WE  BUY  FOR  OUR  TOP  TEN 
CUSTOMERS 


CUSTOMER 

FSC 

#  ACTIONS 

NAVSEA  LOGISTICS  CENTER 
CODE  134  -  N65538 

L099 

30 

L017 

16 

R425 

14 

H399 

1 

R706 

1 

SPAWARS  -  N66001 

L012 

25 

L016 

15 

7010 

9 

7030 

6 

7050 

5 

WHAT  DO  WE  BUY  FOR  OUR  TOP  TEN 
CUSTOMERS 

CUSTOMER 

FSC 

#  ACTIONS 

42 

JWEBL  N3188B 

AD93 

52 

7030 

3 

R706 

2 

AC61 

3 

NAVAL  Al  TECH  DATA  & 

ENER  SERVICE  COMMAND  - 
N32379 

R425 

20 

L016 

14 

L017 

6 

J099 

1 

JTASC  -  N39792 

V231 

11 

R706 

8 

D399 

7 

B599 

6 

AC61 

5 
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WHAT  DO  WE  BUY  FOR  OUR  TOP  TEN 
CUSTOMERS 


CUSTOMER 

FSC 

#  ACTIONS 

NAVICP  -  N00391 

R407 

21 

R408 

8 

R706 

5 

R699 

4 

M299 

3 

EMPLOYEE  METRICS 
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STAFFING 


Series  Breakdown 


i  o  o 
8  0 
6  0 
4  0 
2  0 
0 

■  1  9  /3  0  /9  9 

□  9/30/00 

□  9/30/01 

□  C  urrent 
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TRAINING  S TA  TIS TICS 


120  -r 

100 

80 

60 

40 

20 

0 

“ 

7=^7 

rn= 

f7~T 

FY-01 

Apr 

May 

Jun 

H  0  hrs 

0 

0 

0 

0 

□  1 -8  h  rs 

1 

15 

14 

11 

H  9-24  hrs 

6 

17 

12 

9 

□  25-39  hrs 

6 

6 

11 

13 

□  40+  hrs 

97 

66 

67 

71 
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Budget 

Obligated 

Expended 

15.000 

11.000 

7.800 

V 


16,000 

14,000 

12,000 

10,000 
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Budget 

Obligated 
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7.247 

6.955 
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